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EXECUTIVE SUMMARY 



rnlcssfaculn' nicnilxrs arc in'lliniJ to Icai v tlx* evaltiatioti 
ojteachhi^ to students, who possess only a limited rien\ 
or to cuhtiitmtrators, who often don't hare the time or neces 
sa?y background. th^>n they must he willing to ini vst (heir 
time in efforts in pee ' eraliiation of teachifig(Qm\v<\ 1986, 

p. n, 

'reaching is "ilu* business of tlic business - the activity tiiat 
is central to all colleges and universities" ( Pew Higher Edu- 
cation Research IVograni 19H9. p. i ). But leaching is not 
always taken serit)usi\' anil tt)o often is relegated to a position 
below that of other professional activities. >X'liile there unques 
tionably is superior teaching in the academy, nearly cx en'one 
agrees that it could be inijiroved significantly and that the 
teaching of even llie best faculty could be strengthened. 

What Arguments Can Be Made for Implementing Pro- 
grams to Improve Teaching, Apart From the Personnel 
Decision-Making Process? For Faculty Colleague 
Involvement in Instructional Improvement? 

For decades, academicians haw assumed, usually erroneously, 
that summative evaluation decision making with respect 
to reappointment, promotion, tenure, and compensation - 
is also a means by which instructional improvement can be 
facilitated, hi practice, summative evaluation rarely provides 
sufficient informiuion to laculty for improving teaching. In 
recent vears. in fact, time honoreil practices of faculi>' e\-a 
lualion have been criticized as 'shoddy, intellectually sloppy- 
slipshod, and such a . . . source of shame that it is hardly sur 
prising that leaching is rareU' rewarded in an appropriate wa\'" 
(Scriven 19S0. p. " ). as "'simplistic." ''primitive." and "without 
significant credibility " (Soderberg im. p. 23). and as ' gener 
ally quite limited, sporadic, and inadequate" (Dressel \^r(\ 
p .V3). 

In respon.se to this eriiicism, scholars have recommended 
lhal formative evaluation assessment sj^ecilically designed 
to improve teaching be put into place alongside, but apart 
from, .summative evaluation. Other scholars have suggested 
that tormati\c jx-er e\.iliiation. a proce^-s in which lactilty work 
collaboratively to a.sses.s each others' teaching and to assist 
one another in efforts to strengthen leaching, be developed 
and implemented 

Collaborative peer review [nobabh should include oppor 
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luniiios for Fucully lo icarn iiow to icach more ctTectively, 
lo practice new leaching lecliniciucs and approaches, to get 
regular feedback on their classroom perforniance, and to 
receive coaching from colleagues (Menges 19<SS). Tlie thrust, 
thus, is develop mental a prcKVss providing "data, diagnostic, 
and descriplixe feedback, w ith which to iniprow instrucliori'' 
(Weimer. Kerns, and Parretl 1988) - not judgmental. 

I^eer review should be a comj')onent of formative evaluation 
of teaching because inforn*ied facult\' hax'c knowledge of sub- 
ject matter, teaching and learning, stu den t.s, institutional cul 
ture. and their colleagues* teaching that is uniquely theirs, 
apart from information administrators, icaching consultants, 
and students can prox ide. U is becoming obvious to increasing 
nunibers (.)f f:icult\' that successful teaclu'rs are not only 
experts in their fields of siud\' but also knowledgeable about 
leaching strategies and learning theories and styles, commit 
led lo the personal and intellectual devek)pmeiu of their siu 
dents, cognizant of the complex contexts in which leaching 
and learning occur, and concerned about colleagues* as well 
as iheir own leaching. .'\nd it also is bec'oming increasingly 
obvious lo faculty that they have strengths and weakness(\s 
in ll iese areas, and that many of them could he of assistance 
to and helped by colleagues. 

What Aspects of Teaching Are Facuhy More Qualified 
To Assess than Students and Other Constituencies of 
The Academic Community? 

I-aculty call evaluate their col k 'Agues' performanc e at three 
.stages of in.sirucnon: pre inieraclion. deliveiy, and po.si 
interaction. They can also as.se.ss inierrelaiioiiships among 
I he aforemeniioned .stages and ihe following proce.s.ses: goals 
and objecliws, methods anct materials, and feedback (Soder 
berg 1086). In addition, faculty can crilicjue colleagues' leach 
ii ig on the basis of criteria that are appropriate for this com 
j')lex professional activity According to Scriven, ihere are four 
such criteria: cjualily of content laughl, the in.siructors success 
ill teaching that content and in inspiring .siudeni learning, 
ihe instructor s masicr\" t)f professional skills ir wriiing le.sls 
antl in evaluatiiig ihe acack'mic work of siutk ..s, and the 
in.siructors adherence lo (.'ihical siandardN ( b)8^. p. 30). 

Clohen and McKeacbie's cla.ssificalion of the roles faculty 
shoukl pla\ in assessing (.ol leagues* leaching is a pariicularly 
useful outline ( b)8()). These roles are: elemenis ol' cour.se 



design, including goals, content, and organization; methods 
and materials used in deIi\ei-\' of instruction; evaluation of 
students' academic wc)rk and the instructor's grading practices; 
and integration and inieq^reiaiion of information gathered 
from students, administrators, and self avse,ssment as well 
as from peers. 

What Methods Should Be Used by Faculty to Assess 
Colleagues' Teaching, When the Purpose of the 
Evaluation Is Instructional Improvement? 

A number of '^^i^thods have been employed in formative peer 
evaluation. They include direct classroom obser\'ation, video 
taping of classes, evaluation of course materials, an a,ssessment 
of instructor ex aliiation v)f the academic work of students, 
and analysis of teaching portfolios. Hart has identified six 
instructional events occurring during deliver' that should 
he critiqued by knowledgeable colleagues: 

1. The place where and the time wiien classes are taught 
and other physical factors affecting delivery; 

2. The pnx edures used b\- the teacher in conducting the 
class; 

3. 'Hie teachers use of language to inform, explain, persuade, 
and mc)tivate. and the language students use in respond 
ing and reacting to the teacher; 

-I. The roles i:>la\'ed by teacher and students ;is the\' interact; 
S. Tile relationship of what is occurring in a particular cla.ss 

ro other classes, disciplines, and the curriculum in general; 

and 

(>. *nic outcomes of teaching, as redecled in student learning 
( 198"^). 

Other scholars ( Idbow 19S(); Kat/. and Henr>- 19H8) recom 
mend tiiai dassrcM^m observation be combined with faculty 
i lit er\-iews with individual students or groups of .students. 

\'ideotaping of cla.sses slu^uld be employed for Us unicjue 
jX)tenlial in improving teaching: validating feedback from 
other sources ( Ferlberg ). documenting and preserv ing 
the strengths of teachers, ideniitying weaknesses, and u 'Ui 
paring leaching at different points in teachers* careers ( Licht\ 
and Peterson 19"9). In form;itive peer evaltiation. video phiy 
back feedback should be considered more than m :iltern;iti\e 
to classroom observation. 

Informed peers are idealK suited to ;issess colleagues' 
course materials and evahuition of students* academic work. 
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/Vs McCartliey aixl Peterson suggesu iliese nvaieriais 'prox'ido 
an oxcrview c)f tlic curriculum laugln, infonnaiion about 
icacliin^* strategies, and details about assignments gix'en. \iaie- 
rials can indicate types of communication with students . . , 
the kind of management system used, and resources provided 
to students. . . . There is a plausible logical connection 
i')et\\'een quality materials and ciuality classroom performance 
for many, but not all teachers" ( 1988, p. 261 ). Cohen and 
McKeachie's classitlcation cited earlier is especially instructive 
in describing what materials faculty could examine and how 
to complete what othenvise might be a daunting task ( 1980). 

Sewral program examples in which formative peer eva 
luation has been employed are described in some detail in 
the text. Rea^lers are urged to consider each j^rogram carefully, 
for each has its worthy elements. 

X^liat Factors Can Detract from Faculty Members' 
Willingness to Paitic ipate in Programs Designed 
To Improve Teaching? 

On the basis of the arguments presented thus far it would 
appear that fonnatiw peer e\*a I nation .should be embraced 
for the bettern^ent of the acat^lemy. ^Xe know, howexer, that 
u.sc of this form of in.structional improvement activity has 
been negligible. A number of reasons have been cited for 
ihc unwillingness of faculty to participate in the various 
meihoLls of formatixe colleague a.sse.ssment. 

The disincentives include faculty attitudes toward academic 
freedom: their perceptions of the representativeness, accuracy, 
.md typicality of what is evaluated: their conception o? the 
objectivity oftho.se who conduct the a.s.se.ssment; and their 
\alues with respect to the in.stitution's rewards and incentives. 
Ways fiiust be found to coin ince faculty that what they may 
consider disincentives can be opportunities for professional 
dev elopment, l-or example, having cla.sses obsenvd and mate 
rials as.ses.sed by colleagues for tlie puqiose of in.structional 
impro\ement should no more be considered a threat to aca 
demic freedom than would having colleagues criticjue a pro 
po.sed manu.scripi for publication And including \ ideotaping 
of cla.sses and peer review ofcour.se materials and of instmc 
lor e\aluations of .student.s* aeidemic- work, in addition to 
c lassro{Mn obseiA.ition. should make the process more tred 
ible to the faculty. 
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What Steps Can Be Taken to Enhance Programs 

To Improve Teaching So Faculty WiU Avail Themselves 

of the Programming? 

Scholars insist there are several wa\'S of enhancing the prcx^es.s 
that will improve the likelihood faculU' will develop and take 
part in formative peer evaluation. Besides convincing faculty 
that the **disincenti\'es** can be opportunities rather than lia 
bi lilies, the prcx:ess might be enhanced by involving the 
faculty in the design and implementation of the program, 
in the establishment of standards of effective teaching upon 
which performance will be assessed, in programs that provide 
training in methods of super\'ision and communication, and 
in the inteq^retation and integration of data provided by stu- 
dents, administrators, and colleagues, as well as faculty 
n embers' se I f • assessm eii t . 

How Can Faculty, Students, and Colleges and 
Universities Benefit from Formative Peer Evaluation 
Of Teaching? 

Scholars ha\ e suggested that a number of personal and insti- 
tutional benefits might be realized from faculty participation 
in the fomuui\ e peer evaluation of teaching. Researchers have 
found not only fhatbui /jo// teaching improves when faculty- 
avail themscK cs of programs in which they work collabor- 
atively to impro\e teaching. /Nfter taking part in such programs. 

ll-aci'Jv/ often cicivhp a deeper inUresi in ami commit 
mem to hein^ ci good teacher and, through their exposure 
tif theories of teaching and learning, consciously formulate 
personal philosophies about teaching. For many [facult)i, 
the core of their emerging teaching philosophies is a hum 
anism that emphasizes appreciation of student differences, 
interest in listening to students, and a greater commitment 
to fostering the process of students' intellectual growth than 
to dispensuig hiouiedge. . . . This heightened sensitii it)' to 
students' diivrse needs and challenges, coupled with the 
exploration of teaching and learning theor}', often causes 
/faculty/ to cult irate new approaches to their teaching 
(Austin 1002a. p.HO^ 

Studies also appear to show that tacult\" niorale and col 
Icgiality improve when faculty are involved in formative peer 
evaluation. ^Xhile student learning may impro\'e when faculty' 
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take p'dn in such programs, that is a difficult claiivi to sub- 
stantiate, since many \'ariables besides teaching affect student 
learning. At this time, there is not enough evidence to suggest 
that the tenure status of junior faculty is enhanced wh-n they 
ha\'e participated in formative peer evaluation. 

What Recommendations Emerge From a Study of 
Formative Peer Evaluation? 

Tlie recommendations made at this point are, for the most 
part, general and broad based; other, nK)re specific recom 
mendations are presented throughout the text. 

1. Faculty e\'aluation should include largely separate for- 
mati\'e and summative tracks. Summati\'e e\'aluation. 
including rigorous quaniitati\'e and qualitati\'e data gath 
ering and analysis, is essential for maintaining the acade 
my's integrity; formative evaluation, including equally 
rigorous descripti\'e strategies, along v^'ith. ample feedback 
and opportunities for practice and coaching, is nece^ssaiT 
for improving teaching. 

2. Formative evaluation should include nonjudgmental des 
criptions of faculty members' teaching by colleagues, 
administrators, -dnO where available, teaching consultants 
as well as students; each constituency should be asked 
for data onK' in areas where it is tjualified to pro\ ide it. 

3. Faculty sh(Hiid be encouraged to take part in yearlong 
programs of formatiw peer evaluation of teaching eveiy 
four or five years o\'er the course of their academic 
careers; that encouragement should come from admin 
istrators and the senior faculty. 

4. Faculty should take leadership in the design and imple 
mentation of programs of formative evaluation of teaching. 

5. I-acult\' should be pro\'ided opportunities for training in 
the skills needed to conduct formative peer e\'aluation. 

6. The involvement of the faculty in the formati\'e evaluation 
of leaching should be guided by expertise in appropriate 
areas of the knowledge base of teachitig; at the same time, 
care must be taken lo minimize potential problems thai 
can ari.se from ha\'ing the same faculty in\'olved in the 
format' ve and sutnmative evaluation of a colleague. 
Formative peer evaluation .should include diieci cla.s.sroom 
obser\'ation, videotaping of chisse.s, evaluation ofcour.se 
materials, an assessment of instructor evaluations of the 
academic work of .students, and analysis of leaching port- 




folios. In this prc^ess, it is advisable to use the methods 
in combination, not as independent entities. 

8. Institutional rewards and incentix'es should be structured 
to demonstrate to faculty that participation in formative 
peer evaluation of teaching is truly valued. 

9. Research should p.rt^eed along sexeral potentially luc- 
rative lines: the interaction of variables in specific insti- 
tutional contexts: the tie between participation in for 
matixe peer evaluation and motivational theor\'; dcKu 
mentation and reporting experiences with formative peer 
assessment, whether they are succeeding or have failed; 
and rigorous empirical and ethnographic study of pro 
grams of formative peer evaluation currentK- in place. 
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FOREWORD 



U is a contrudictiDn iliai. in an organi/.aiiDn in whid. the pri- 
nvaiy activity i.s inMruction. Licvciuping \hc .skill of teaching 
and the recognition of goiKl teachers is so undervalued. This 
contradiction has not been lost on the general iniblic but. 
because ofthe lack of hard evidence to the coniraiy. the con 
ventional opinion is that this contradiction seems to have 
made only the slightest dent in the consciousness of many 
academic IcLiders. In reality, this is not the case. Mom. if not 
all, of higher education's academic leadership are abundantly 
aware of the imj^oriance of good teaching and the lack of edu 
cation that graduate schools give their students concerning 
the research and practice of teaching. This aw areness is not 
translated into specific results not because faculty lU) not 
want to be g(;od teachers, but bei'atrse ofthe way academic 
leaclership iias allowed the arguments to be framed by the 
professorate and the lack of genuine rewards tha' the lead 
ershij^ provick's forgoo^l leaching. 

The primary barrier as .seen b\- academic leaders is the re 
strictions uiused by the concept of academic treedom re 
inforced by tenure. On the surface it ai^ix*ars that the shield 
of acailemic freedom is u.sed by a fai'ulty who want abst)lute 
freedom and minimal accountability. W hile faculty want to 
be protected from arbitraiy actit>n thai will inhibit the free 
flow of ideas. sur\ e\ s clearly show that they want to be good 
teachers. 'Che .shield of academic freedom is u.sed because 
there is a genuirjc fear c-oncerning the u.se and accuracy ol 
the e\aluation.s. 

'I'h.ere are two separate and di.stinct bases for this tear. Tlie 
first is faculty's experience with evaluatit)ns. Almo.st alwa\s. 
faculiv have seen e\'at nations conducted to ju.stily .something 
negaiixe. Therefore, mo.st evaluatii)ns are used for criticism 
or to iu.stif\- toj) ck)wn admini.sirati\e acti')n. A second concern 
relates to the accuracy of evaluations of leaching. Mosi taculty 
reali/.e they cK) not ha\e aclec|uate formal training [o be out 
standing teachers. What ihcy haw learned they have gained 
through modeling faculty they had as .students and through 
their informal, on the i« »b. hit ov nuss teaching experiences. 
Since they know they do not ha\e tlu' ex|H'rii.se to iudge good 
teaching ihem.selves. there i.s real doubt thai admini.stralors. 
w ho ar' • e\en more uniramed. w« »ulcl ha\e belter skills to eval 
uate teaching. 

These two concerns are reinforced by the lack ot action 
by academii leacler.shiji to sjx'i ilically imi^ove the atnu) 
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splicrc and conditions surrounding (he o\uluaiion oru*a<.iiini». 
Tiio concerns could ho addressed in several ways. 

• Recjuire all d(KU u'al sludenis lo receive uainini^ in ihe 
research and skill oCieachini^ in their coL^niiiw area. 

• Make it jxui ofihe ()ri»ani/.aiional culuire ihai all t'aculiy 
continuously w ill lake courses aixl rec<.'i\e irainint^ to 
lmjM*o\ e iheir kn.owiedge and skill ofieachini^. 

• Provide iiiceniives, .such as salary increments or bonuses, 
lo encourai^e faculty to s(.'ek ceriification of inert. 'a.sed 
leaching skills. 

• Make all e\aluation purely a iorinatiw or dewiopnienial 
process, thus eliminating the tear of improper u.se oi" 
teaching e\aluation. 

/Vs.serting tliis type of major change m the way an institution 
addres.ses the importance of demon.straied teaching skill in 
its faculty cannot happen overnight, liowx'ver. it is di.stinctly 
probable that private atul public supports of higher edticaticMi 
will begin to demand such changes Many institutions have 
taken steps to prepare by making small, nonthreatening 
changes that will begin to [^ositiwK alTeci (he teaching culture 
of their instituti(.)n.s. 

*i his report, written by l;iriy Keig. at^ljunci jM'ole.s.sor. and 
Michael I). Wag^goner. as.sociatt.* profe.s.sor cMkI hea(.l of the 
department of (.xlucational admini.straiion and counseling at 
the I'niversity of \orthem Iowa Ciollege of l-<.lueation. (.-Nam 
ines ways in which institutioiis are affecting their teaching 
climate. Through the tise of jxvr rt^'view of leaching, faculty 
involuntarily are seeking to improve their instructional skills, 
■fhe authors begin by establishing rationales for formative 
evaluation, for peer review in instructional imjMtAement 
efforts, and for a program of conijMehensive faculty evaluation 
in which formative pet^M' asst^'ssnKMU is an (.*ss(.*ntial comj'jo 
nent. They then examine the roles that might be played by 
jU\Ms in a.ssessing (.oiU^agtu.' t(.'a(. hing and fi\(.* nKiho(.ls |x\'rs 
haw uset^l to a.ssi.st colleagues in efforts to improve in.struc 
tion. After (.'xaniining .sevenil prt)grams in\ol\ ing differ(.'nt 
meiho(.ls of p(.^M- re\i(.A\ atxl factors that (.'itlkM- (.k'trai^t from 
or (.MX(nirage f;Kulty willingness to jxiriu i|Xit(.' in this f'oini 
ol e\alnanons. the authors cone hide with a number of ret om 
meiulations foi the prat^ tu of fonMati\(.' pi.xM (.valuation of 
teaching 
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The pressure on higlicr education U) nK*asural-)ly dcnion 
straic aiiiviiy is increasing. Iligiicr education is hect)niing 
more iniporiani to society and, therefore, .society is les.s wil 
ling to alK>\v "Inisiness as usual" to continue, institutions may 
anticipate these dcman^ls and establish their own ways \o 
develop e\ idence that they are being resjionsi\e to the.se 
expectations, or they can wait for external areas to toix\ their 
measureineni retjuirements. Demonstrating that tacult\ teach 
ing skills continuously are being improved is one of the ways 
institutions can demonstrate their responsiveness to ensure 
the distinction of iheir institution. Tliis report provides an 
excellent foundation for tho.se institutions that see peer eval 
nation of faculty as one process that can help to make the cul 
lure more responsive lo the evaluation and improvenieni ot 
leaching. 

Jonathan D. Fife 

Series Kdiior, Professor i)f Higher I'.ducalitMi .Vdministration 
and Director, I-KIC. Clearinghouse t )n Higher Mducaiion 
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INTRODUCTION 



Teaching is a complex web of acts, a fact to whicii those of 
us who stand in front of students in classrooms or who mte- 
ract with students in other ways can readily attest^To he val- 
uable to f£cult%s the ex'aluation of teach.ing must be sophis- 
ticated. s>-stematic, and thorough, as well as flexible enough 
to .-apture the substance. e.ssence. and nuances of the teaching 
process For years, the primary emphasis in faculty ex-aluation 
has been on decision making relatix-e to reappointment, pro- 
motion, tenure, and compensation. In most instances, pro- 
fessors- effecti^■ene.ss as classroom teachers is but one clement 
considered in making these judgments. Generally, it is doub 
Ril that the personnel decision-making prcxess has had much 
positive impact on improving leaching (Crcxss 1986; Dre.ssel 
19^6; Hodgkinson 1972; Scriven 1980; Weimer. Parrctt. and 

Kerns 1988). . 

m assessing teaching, decision makers usually rely hcax 
ilv-oftcn too extensiveK--on student ratings of courses and 
instructors. We believe that when faculty and administrators 
alk)w student ratings to be the only real source of information 
about teaching, thev unwittingly contribute to a -SN-stem in 
which too much emphasis is placed on evaluating superficial 
teaching skills and not enough is placed on more substantix e 
matters: N4oreover. we believe faculty and administrators abne 
aate their professional responsibilities when they are unwiU- 
iiig to as.se.ss aspects of teaching they are particularK' well 
qualified to evaluate. 

We a'-guc here for a process for evaluating teaching that 
has as its end improvement of instruction -one m which 
f-icultv peers (and other qualified con.stituencies) provide 
information their colleagues might use to become better 
teachers. Xvx: are convinced that if facultv' desire crcdib c exal 
uation, thev mu.st create largeK' independent .sy.stcms that pro 
mote teach'ing improvement and contribute to rational per 
.sonnel decisions 

in this report, we look at how evaluation might he used 
for iiLstructional improvement and. more specifically, at how 
peer lex'iev^- might be used for this puqxisc. First, we atlcmi^t 
u, establi.sh rationales for formatix'e evaluation, tor peer review 
in in.structional improvement efforts, and for a program oi 
coniprehensixe faculty evaluation in which formative peer 
evaluation is an e.sseniial component. Second, we examine 
the roles that might be played by peers in a.sse.ssing their col 
leagues- teachir.g. Tliird. we look at fn c methods in which 
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peers lia\'e been im'oh'ed in efforts to imprcn-e tlie teadiing 
of tlieir ccMeagues. Fourth, we describe programs involving 
peer review. Fifth, we consider factors that niav detract from 
faculty' members' willingness to participate in this form of 
evaluation. Sixth, we examine factors that may improve the 
likelihood (^f faculty in\'olvement and enhance the process. 
Seventh, we consider how their pa.i'cipation ma\' affect teach- 
ing, learning, and faculty morale and collegialitv.' Finally, we 
present a number of recommendations for the practice of for 
mative peer evaluation in higher education. 

State of College Teaching 

Teaching is, it has been said, "tlK.' business of the business" 
of higher education. We al.so are ;oltl that: 

Too seldom w collegiate teaching Hewed for what it is.- . 
the act it 'ity that central to all colleges and unit vrsities. 
Teaching is the task that distingnL^ihes colleges and univer 
sities, along with primary and secondary schook. from all 
other serrice agencies (Pew 1989, pp. 1.'2). 

In mam- colleges and uni\-ersities. teaching anti related actix'- 
ities are subordinated to otlier academic responsibilities espc 
cially research and publication (Bover 1987; Carnegie Fciun 
dation 1990a, I990h: Fainvcather 199^: Uidd 19^9- Miller 
1990; Pew 1989, 1990; ScriNvn 1980; Soderberg 198=;; Stuck- 
C.roup 1984). Calls are being made b\- go\ ernment offieials 
b\- the gen.eral public, in the popular press, and from within 
the academy itself to establish once again the role of teacliing 
as the primary- responsibility of eolleges and uni\ ersities. 

B\- nearly all accounts, teaching in colleges and universities 
IS not ;Lsgcx)d as it should be. -niere are. no doubt, man\- rea 
sons to explain why faculty do not alwavs teach as well as the^• 
could. We di.scu.ss six factors that hax e been cited as rea.sons ' 
for less than effect i\e teac-liing. 

First, it is diffieult to refute claims that teai-hing has been 
neglected by man\- pri)le.s.sors. due in part to a reward .striu' 
ture that puts an undue premium on traditional .scholarly acti\- 
ities anti devalues teaching, ^et. e\en at liberal arts colleges 
where teai-hing is the preeminent facultv role, there .still is 
plenty- of poor teiching. Clearly, then, not all of the blame 
for ineffective teaching can be laid on the acadeniv's reward 
structure We return to this i.ssue in the .sections on disincen 
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lives and incentives. 

A second factor is the erroneous assumption '^that teaching 
a subject matter requires only that one know it" (Eble 1988, 
p. 24). A number of voices have responded to this of late, 
among them this one, noting that: 

Advanced knowledge of a subject is not itself a sufficient 
preparation to teach students. . . . Advanced degree holders 
or candidates are largely content with believing that one 
simply teaches by doing, an attitude of being somehow 
above conscious pedagogy' 1990. p. 5). 

Said another \va\': 

A body of knowledge us essential. But it is of limited value 
without . . . an understanding of hoiv learning occurs . . . 
The success of the classroom enterprise depends both on the 
teacher's conununication of knoivledge arui his or her abil- 
ity to help students construct their own knowledge It de^ 
pends on the teacher's skill in encouraging dialogue and 
in probing students' underi>canding: provoking questioning, 
guiding, and interpreting (Pew 1989, pp. 3 4). 

A third, more basic factor that may explain why teaching is 
not as good as it could be is the mistaken nouon that teachers 
are bom and not made. Shulman insists that a successful 
teacher learns how to teach, and continues to leam, by study- 
ing not onl\' in a specifu; content area but also in fields related 
to teaching (history*, philosophy, psychology, and sociolog\', 
for example) and in the liberal arts ( 198^, 1989). Tlie general 
areas in which teachers need to be well-versed in order to 
promote student learning are: 

• Content knowledge; 

• General pedagogical knowledge, with special references 
to those broad principles and strategies of classroom man 
agement and organization that appear to transcend subject 
matter; 

• Curriculum knov^ ledge, with particular grasp of the mate 
rials and progran is that scr\'e as "tools of the trade" for 
teachers; 

• Pedagogical content knowledge, that special amalgam 
of content and pcdagog>' that is uniquely the province 
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of teachers, their own professional form of professional 
understandings; 

• Knowledge of learners and their characteristics; 

• Knowledge of educational contexts, ranging from the 
workings of the group or classroom, the governance and 
finance of [education], to the character of communities 
and cultures; and 

• lOiowIedge of educational ends, purposes, and values, 
and their philosophical and historical grounds (Shulman 
1987). 



Acquiring breadth and depth of knowledge in these arc:LS will, 
Shulman says, enhance the likelihood that faculty will develop 
the competencies that will allow them io communicate, with 
increasing sophistication, what they knov to their students. 
The competencies, which are influenced by all the teacher 
has experienced as well as virtually everything he or she has 
studied, encompass the teacher's abilities to: 

• Comprehend both content and purposes (p. 15); 

• Transform the content knowledge he or she possesses 
into forms that are pedagogically powerfiil and yet adap- 
tive to the varieties in ability and in background presented 
by the students (p. 15); 

• [Organize and manage) the classroom, [present) clear 
explanations and vivid descriptions, [assign and check] 
work, and [interact] effectively with students through 
questions and probes, answers and reactions, and praise 
and criticism (p. 17); 

• Check for understanding and misunderstanding . . . while 
teaching interactively, as well as [by] more formal testing 
and evaluation that teachers do to provide feedback and 
give grades (pp. 18-19); 

• (Reflect on) the teaching and learning that has occurred, 
and [reconstaict, rcenact, and/or recapture] the events, 
the emotions, and the accomplishments (p. 19); and 

• Arrive at a new beginning . . . with nc\w comprehension, 
both of the purposes and of the subjects to he taught, and 
also of the students and of the prcK'Csses of peclagog\' 
themselves (p. 19). 

Also recognizing that subject matter master^' is hut one C()m 
petency required of effective teachers, Yarbrough lists a range 
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of capabilities, skills, and behaviors that are essential for devel- 
oping expertise in teaching ( 1988). Tliese competencies, or 
"dimensions of knowledge," are: 

1. Subject-matter content knowledge 

2. Subject-matter pedagogical knowledge 

3. Curricular knowledge 

4. General pedagogical knowledge 

5. Knowledge of lean:er characteristics 

6. Kiiowledge of commui:icaiion techniques 

Yarbrough goes even further, noting that: 

// is not enough to say that expert teachers possess expert 
knowledge in the content of their subject matter, inpeda 
gogical pnnctpk% in principles of learning, and in specific 
knowledge of their studeyits' current knowledge, capabilities, 
and limitations. It also requires that expert teachers have 
the capability to process information from multiple dimen- 
sions in concert and to apply that information to new and 
potentially unique problems in the teaching process {p. 223)- 

YarbrouglVs "dimensions of knowledge" are not too different 
from Shulman\s '*pedagog\- of substance*' ( 1987). Yarbrough, 
in addition, explores an experiential dimension as well, 
ol\ser\'ing that: 

Expert teachers (based on peer nominations) are more likely 
to perceive a ren brief sequence of events in a classroom 
as supplying a wealth of information to be interpreted in 
reference to iwious domains of stored knowledge. Begin 
nifig teachers i iewing the same sequence of events inter 
preted the eivnts in a unidimensionalfashion. It appears 
that expert teachers have autornatic procedures that ref- 
erence teaching events, eivn at the procedural letxi to more 
diverse domains of knowledge than do beginning teachers 

(p. 22-n. 

Clearly, there is more to effcctiw teaching than having a com 
mand of subjcLi matter, as essential as that is. vx^ays must be 
found, through graduate school training and inser\ ice pro 
gramming, to help teachers de\elop other requisite 
competencies. 
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A fourth factor likely to limit instaictional effectiveness is 
a teacher's unwillingness to tai<e differences among students 
into account. As Katz and Henr\' obser\'e: 

Though students are not all alike, teaching continues on 
the assumptiofi of a common denominator that, when 
found will enable the faculty member to reach most of the 
students Those faculty' who hare giivn up aiming for the 
middle range of their stude) its hare resorted either to 
addressing primarily those few students who seem likely to 
catch on to their particular perspectire and lerel of thinking 
or to focusing on those sti/dents ivho ?ieed the most help to 
make it through the particular course. So?Ne faculty hare 
giivn up altogether on detennining a strateg}' for coping 
with student differences and simply rini their course on 
a si?ik'Or swim basL^^, counting on the "objectivity'' of their 
grading s}'ste?n to maintai?i "sta?idards'* and thi/s aividing 
the fact of St Indent diversity As the numbers of st indents in 
college classrooms hare risen or er the years, the probabilit}' 
and the possibility of atte^uiing to student differetices hare 
decreased. \X'hen classes are small, and some gire and take 
dLscussio?i is encouraged between the teacher and the stu 
dents, It is more difficult to overlook differetices among stu 
dents But with koge classes and the use of the lecture 
method, the fact of individual differences is nna h less evi 
dent and these differences can he rather easily ignored 
{ im p. 1 11 ). 

A fifth factor that ma\- limit more effective leaching is, accord 
ing to Shuiman. the locus from wiiich teaching improvement 
efforts often emanate ( 1993)- He contends that programs 
initiated by iini\'ersii\vvide "centers," w hile praisewortiiy in 
many ways, may militate against making teaching "community 
property" in which decisions about "quality, control, judg 
meiit. evaluation, and paradigmatic definitions" rightfully are 
made !.y members of each community (namely, faculty in 
the various academic lields on campus) (p. 6). Shuiman con 
lends that: 

W'c need to reconnect teaching to the disci/)lines Although 
the disci/)lines are easy to hash because of the nianyprob 
lems they create for us, they are, nevertheless, the basis for 
our intellectual a tnnn unities. I. ike it or not, the forms of 
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scholarship that are seen as intellectual work in the disci- 
plines are going to he rained more than the /onus of scho- 
larship (like teachi?ig) that are seen as non disciplinary'. . . . 

Institutional support for teaching and its nnprorement 
tends to lie in a unirersit}' center for teaching and learning 
. . , where faculty^—regardless of department—can go for 
assistance in improring their practice. That's a perfectly 
reasotuible idea. But notice the message it conixys — that 
teaching is generic, technical, and a matter of performance: 
that it's not part of the coynmunity that means so much to 
most faculty^, the disciplinary, interdisciplinary, professional 
community. It's something getu^ral you lay on top of what 
you really do as a scholar in a discipline. . . . \Xe need to 
make the rerieu\ examination, and support of teaching the 
responsibility of the disciplinafy community (1995, p- 6). 

Finally, as Miller reniiiuls us, "Tliiit [vvhilel good teaching 
is often a nialler of instinct as it he personality, it is also a mat 
ler of hard work: discipline, perseverance, and the insight aixl 
inspiration that derive from intense moments or hours of cele 
bration" ( 1990, p. 59). Or, ;ls Mengev more bluntly puts it, 
"Some faculty resist working hard on their teaching" ( 1980, 
p. 2-). 

There are many ways in which teaching could be improved. 
Too few teachers have examined in a systematic wa\' how they 
teach or ha\e thought about it seriously or reflectively. Aixl 
they rarely get the help they need to improve from those who 
are in the best position to proxide it: informed peers, teaching 
ct)nsultants. and administrators. In this report, we hope to 
establish our premise that our colleagues haxe uniciue abil 
ities. apart from consultants, administrators, and students, that 
can help us in^pro\ e our teaching. 

State of the Evaluation of College Teaching 

Arguments for and against the usefulness of evaluations of 
college teacheis and teaching, the \'alue of differei t methods 
of evaluation, and the suitability of \ arious constituencies in 
assessing the instructional etTecti\ eness of faculty ha\ e been 
(x;currii>g for dc(.*ades. Regardless of the debate, faculty have 
been and continue to be evaluated, formally or informi ily, 
by individuals or grt)ups of people, lliese judgments affect 
faculty members' personal and [uofessional relationships with 
students, administrators, and colleagues and often also their 
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reappointmeni, promotion, tenure, unci conipensaiion. 

Cohen and McKeachie ( 1980) and Scrivcn ( 1980) argue 
that it is in the best interest of the faculty' to have a formal, 
systematic program of faculU' evaluation in place, because 
it can protect faculty- from unjust personnel decisions. Ixe 
( 1982), Smith ( 1981), and Stevens ( 1985) suggest that such 
a program is also in the best interest of colleges and univer- 
sities, since it can demonstrate that the process has been 
handled rationally, rather than arbitrarih' or caoriciousK', when 
personnel decisions unfavorable to faculty are litigated. These 
recommendations refer specifically to summati\'e evaluation- 
decision making relative to reappointment, promotion, tenure, 
and compensation— which emphiLsi/es judgments about, not 
improvement of, performance. 

Whether carried out formally or informally, effectively or 
ineffecti\ely, summative evaluation has been, and remain,s, 
the principal method by which to e\'aluate teacher,s. For a long 
time, it has been assumed that this t\pe of evaluation also 
would function instaimentally to improve teaching. But the 
notion that there is a direct relationship bciween summative 
evaluation and in,structional improvement has been chal 
lenged in recent years. As indicated earlier, there is wide 
spread agreement that this time-honored practice of evalua- 
tion has not significantly improved teaching. \n decr\1ng 
prevailing practices of faculty evaluation, Scriven has witten 
that these approaches often are ,so '\sht)ddy, intellectually 
sloppy, slipshod, and such a . . , source of shame, that it is 
hardly surprising that teaching is rarely rewarded in an appro 
priate way" ( 1980. p. "'). lx\ss acerbic, but eciually critical, 
ScyJerberg de.scribes usual apprcxiches as "simplistic" ( 1986, 
p, 13). 'primitive/* and "without significant credibility ' (p, 
23). Similarly, Drcssel refers to common practices as 'gener- 
ally quite limited, sporadic, and inadequate" ( 1976, p, 333). 

Assessment emphasizing instructional improvement has 
been derived recently in high.er education. NXliile interest in 
this form of evaluation is perhaps at an all time high, the inter 
est has waxed and waned owv the years. All the while, how 
ever, influential .scholars in the fields of teaching and student 
learning ha\'e continued to piXMiiote the idea that fomiativc 
evaluation a,s.se.ssment specifically designed to imi-jrove 
teaching--.sh()uld beccMiie a regular part of the process of 
evaluation (Cro.ss 1080; Dre.s.scl 19*^6; HodgkinsDn \^ri\ Kat/ 
and Henr\* 19H8; Sorcinelli 198-1; Weimer 1990). 
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Motivating faculty to improve their classroom teaching may 
be positively affected to a degree by rewards dispensed or 
withheld through sum.mative evaluation. But it also is said 
that extrinsic rewards (such as promotion, tenure, and salary 
increitses) simply reinforce, rather than improve, performance 
(Hodgkinson 19*72) or even reduce faculty motivation to teach 
effectively (McKeachie 1982). And the common faculty com 
plaint that summati\'e evaluation does not adequately assess 
their performance also may diminish its value in improving 
their teaching. 

Proponents of formati\'e evaluation are convinced that it 
is more promising than summative evaluation for motivating 
faculrv' to impro\'e their teaching. Scholars also suggest that 
programs of instructional improvement should be a much 
more important institutional priority. Rationale for formative 
evaluation, for peer involvement in formative evaluation, and 
for a comprehensiw program of faculty evaluation are pre- 
sented next. 
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MAKING THE CASE 



The impc)rianccM)f making fair and impartial personnel deci 
sions cannot he disputed. Vet, decisions of this type are made 
only periodically. usuall\' relatively earl\' in a faculty memher\s 
career. Surely an et|ually if not more compelling reason for 
evaluating courses and instructors is to improve performance 
in progress. If a goal of evaluation is instructional impro\c 
ment, it prohahly will be i .ecessary, as Licata suggests, to e\'al 
uate teaching, research, and ser\'ice not just when personnel 
decisions have to be made but more regularly throughout 
faculty membei-s' careers, early in their appointments and at 
various midpoints and later in their tenure ( 1986). By doing 
so, colleges and universities might demon.strate more con- 
vincingly to .students, parents guardians, faculty, and other 
con.stituencies that teaching, teaching improvement, faculty- 
growth, and faculty development are important in.stitutional 
priorities. 

Tntil recently, information about the effectiveness cf 
teachers has come almo.st exclusively from student evaluations 
of courses and instructors. At many -perhaps most --colleges 
and universities, reliance on .student ratir ;s for this informa- 
tion is .still the rule rather than the exception. The idea that 
other con.stituencies. especially a faculr\* member's peers and 
academic administrators, may have rele\'ant information about 
faculty members' teaching has only begun to be explored. 
Data provided by each of these ct>nstituencies may be unique, 
all of it neces.saiy if significant in.struclional improvement is 
to tKvur. 

Inasmuch as there are different puq^oses for evaluating 
faculty (tonnaiive. summative, and perhaps others) :ls well 
as sewral constituencies who can a.s.sess specific iLspects of 
faculty performance, there seems to be a need for more com 
prehensive faculty evaluation. In this .section, we look more 
closely at rationales for formative evaluation, for peer review- 
as one element in that pr(KV.ss, and for comprehensive faculty 
evak:ation. We also look at three moulds that involve forma ' 
live peer evaluation. 

Rationale for the Formative Evaluation of Teaching 

In higher education, faculty evaluation is carried out primarily 
for decisions regarding reappointment, promotion, tenure 
and compensation and only .secondarily, if at all, for improving 
performance By necessity', all facility r(')les -research and ser' 
vice as well as teaching are accounted for when personnel 
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decisions arc made. During these deliberations, faculty usually 
are e\'aluatcd according to common standards. 

In formatix-e e\-aluation. the emphasis is on development, 
where efforts toward improN'ement can be directed towaixl 
anv or all professional roles. If so desired, emphasis can be 
placed on insta.ctional improx-ement. and faculty differences 
in educational philosophy and teaching style can be taken 

into account. . , 

In commenting on instructional development within a 
broader context of faculty development, Menges point^s out 
some of the benefits of assessment expressly designed tor 
teaching improwment; 

IMvlopnwnt fisl a natural process, an imjoldin^ grMjual 
and coutinual. ilt is/ fmhly individual: it proceeds di J jcr- 
cntlv from person to person and from setting to setting. Its 
initiation « usually from within, although certain external 
circumstances support it more ejjectively than others . . . 

We hare had some success in identifying conditions under 
which facuHv members are likely to learn about and adopt 
new instructional approaches. . . These conditions include 
exposure to rekwnt theory, proi mon for practicing the new 
approaches and receiring feedback on the practice, and 
opportunities to he coached while applying the new 
approaches in the field. (Note the happy coincidence that 
the^e conditions for faculty' are similar to conditions which 
promoto learning and transfer for students in our classes ) 
(198S, pp. 181 82) 

Provisions for learning about teaching, practice in implement 
iuR ncu' approaches and techniciucs. feedback to faculty on 
their efforts, and coaching from colleagues or consultants nor 
malK' are not part of the prcxess of tliculty evaluation, al 
though increasing numbers of scholars and practitionei^ 
believe there should h.c meiMi;, available for doing so. V^ nile 
there mav be resistance from laculty to involve themselves 
,n another professional aciixity ( Menges 198S), there probably 
•ire waN's to motivate laculty to take acK'antage of formative 
cx aluaiion of this tvpe. although the type of motivation ver>' 
uell mav differ am<mg faculu members ( Bess 198. ). 

M im' scholars critici/.e the lack of relationship benvecn 
ficullN'evaluation and teaching imprcAcment. 'Hiat sentiment 
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is expressed sua inciK* b\- Aubrecht. who says. "Verv* few insti- 
tutions are making good use of their faculty evaluation s\'S- 
terns for de\'elopnient purposes" ( 1984, p. 88). 

A number of scholars now recognize the need for formative 
as well as summative e\:jiuation. Most of them recommend 
that the two functions be kept distinctly separate. Others 
believe that information gathered in summative evaluation 
also can be used for formative evaluaticjn. Scholars almost 
unanimously agree that it is unwise and counterproductive 
to use information collected in formati\'e evaluation in making 
personnel decisions. 

The view that summative and formative evaluation should 
be separate entities in the assessment prcx*ess is a common 
one. 'I'liis notion is expressed in two pieces c)f recent schol- 
arship, in a particularK' strong voice. Cross describes funda- 
mental differences betwcc!! formative and summative evaluation: 

/rotu'cally. praciiccilly all the proposaLi and practices in 
ci:^esiiit)W)it ti)clciy innilve summative evaluation. . . . There 
are Jeir proposal for formative evaluation to show us how 
to improve educatio)i in process, . . . If we are to improve 
the ijucdity of education, perhaps the most important ques- 
tion . . . to address is what decisions should he made to 
improve instruction. . . Hoiv students are taught lies at the 
heart of (/luilit}' education. It makes the difference betweoi 
a lifelo)}^ learner and a grade grubber, between enthusiasm 
for learni}ig and indifference to it, between an educated 
society a/?d a credentialed o)u>{ 1986, pp. 3 4). 

\veimer. Kerns, and Parreit are more direct in explaining why 
summative and fomiative evaluation require different types 
of information, suggesting why instructional improvement 
is unlikely to occur from summative evaluation; 

// is the i)ite)U of summative evaluation to provide the com 
parative data for subsetjuent use in perscnnicl decisions. 
These evaluatiotis typically consist of items that describe 
teaching in global terms . . . Whereas the intent of forr/ta 
five evaluation is to provide data, diagnostic and descriptive 
feedbuik'. with which to improve instruction. . . 

The purposes of the evaluations are different. Consequently 
the items on fornuuive evaluatiofi instruments and thi items 
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summathv iTaliicitiofis should clijjcr als(K . . . Whcti 
items do Not correspond with ihc intended pnfposes of the 
eralncition, e.g., proriding sumnuitire data irith the expec 
tation that the data trill be used to iniprore, the potential 
value of the eraluation process is attouiated ( 1988. p. 286). 

Thai admonition nomiihsianding, there are those who argue 
thai data gathered for summative exaluation can he used for 
instructional improx ement. DresseL for example, suggests 
that "assessment activity must he hroadh* conceived as a l")asis 
for improvement, not for making personnel decisions. Eval 
uation can be Hnked to reward structures, but with recognition 
thai improvement and dexelopmeni are the firsi concerns " 
(19"6. p. 374). 

Writing alone a \'ear before collaborating wilh others (Wei 
mer, Kerns, and Parrett 1988), VCeimer seems to .strike a posi 
tion somewhere between complete .separation of summative 
and formative evaluation and a linking oftho.se functions. In 
that provocative piece, she recommends: 

Separate formative afid summative evaluation activities, 
hut litih the results The two activities should run on separate 
tracks with paints of convergence at the beginning and end. 
Summative evaluation constructs the comprehensive picture 
of nistntctional conipetency. Formative evaluation closes 
in on the picture, dissects the c(miponent parts, analyzes 
the relatiotiship to one another, identifies what parts should 
he changed, and provides initial feedback on the succes'i 
of those changes Summative evaluation occurs again to 
create another composite picture, this time to show the dif 
ferences The connection between the tiro cannot he over 
emphasized. Formative evaluatiofi must target appropriate 
areas of change. Summative assessment must reflect the 
impact of those changes { \9>P. p. 10). 

Rationale for Peer Review in the Formative Evaluation 
of Teaching 

Several scholars beliexe that college teaciiing could be im 
|m)ved signitlcantly if faculty worked collaboratively to thai 
end. Wliile acknowledging there arc .still tho.sc who think 
command of subject matter is all that is recjuired to teach, 
scholars .say that faculty are becoming incrcasingK- aware thai 
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successfiil teachers are knowledgeable about leaching strate- 
gies and learning thet)ries, committed to the individual devel- 
opment of students, cognizant of the complex contexts in 
which instruction cKCurs, and concerned about colleagues^ 
teaching as well as their own. Ever\' teach'^r has strengths and 
weaknesses in these areas, and many of them could assist 
and, or be helped by colleagues. All of this suggests t' t good 
teaching is developed over time and rarely is ftjlly actualized. 
In short, college teaching will improve when faculty support 
each other with expertise that is uniquely theirs, apart from 
what students, teaching consultants, and academic admin- 
istrators can contribute to instaictional improvement (Batista 
1976; Centra 1975, 1986, 1993; Cohen and McKeachie 1980; 
Hart 1987; Mathis 1974; Seldin 1984). 

A number of the academy's conventions—epistemological, 
cultural, political, and practical factors that may undermine 
efforts to improve teaching— underscore a need foi formative 
evaluation and for peer involvement in that process. These 
conventions include: the lack of agreement on what good 
teaching is; the inflexibility built into most sy,stems of eval- 
uation; the isolation of profe,ssors from one another with 
regard to teaching; the reluctance of faculty to seek help from 
colleagues with more expertise and experience and with 
higher status; and a reward structure in which research is 
more valued than teaching. 

Abrami ( 198S), McKeachie ( 1986), Smith and Walvoord 
(1993), and others have obser\'ed that scholars do not agree, 
and probably never will, on what good teaching is and on 
how to evaluate it. It is not suiprising, then, that faculty 
develop idiosyncratic teaching st^-les based on personal pref 
erences and normative' assumptions rather than on a uniform, 
codified standard of exemplar)' practice (Bulccx'k 1984; 
McKeachie 1986). For that reason, informed peers— who 
know their colleagues personally and professionally, are famil- 
iar with effective and ineffective practice in specific fields of 
study, and recognize that effective teachip.g is contingent upon 
a number of complex factors, onlv some of which can be con 
trolled by instructors— may be especially forceful catalysts 
in the prcxress of instructional improvement (Mathis 1974; 
Soderberg 19R6). 

Because teaching is so complex, scholars often criticize 
the rigidity of systems of evaluation based upon superficial 
style characteristics. In envisioning a more flexible alternative. 
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Qmcelli proposes u system of assessmeni that: 

makes mitiinuil assiwiptiom regarding how iristructioti 
should occur. It is left to the professional judgments of the 
professors to dete?ini?ie how they wish to dei vlop and teach 
their courses The system only requires that they be withi^ 
the bounds of acceptable practice, broadly defined, that they 
do what they say they do, and that there be a cogent ration- 
ale for their choices. Thus, the review of each professor is 
unique and requires decisions based on disparate and often 
idiosyncratic bits of information. The use of judgments by 
peers provides a method that is flexible enough to adjust to 
the unique data base generated in each review ( 19H7, 
p. 12). 

It is relatively uncommon for teachers at any le\'el— whether 
in elementarN' and secondarN^ schools or in colleges and uni- 
versities— to work collaboratively in improving teaching 
(Copeland and jamgcx'hian 19H5; Johnson, Johnson, and 
Smith 1991; Katz and Heniy 1988; Shulman 1993). though 
collaborative relationships probably are more common in the 
K- 12 sector than in higher education. Copeland and jamgo- 
chian note: 

The usolatio?! of teachers to meet together outside the ckdss 
room is a well- documented phenomoia, . , . ft is rare for 
two teachers to meet together outside of class time to discuss 
substantive issues related to their students or their own 
teachifig. Systematic analysis of teaching exploration of 
alternative approaches, arialysis of individual teach i?ig a? id 
learn i?ig problems, a? id the generatio?i of a? id testing of pos- 
sible soli4ti()?is are all activities that typically occur at the 
individual teacher level, not among colleagues (p. 18). 

CA)pcland and Jamgochian suggest that a number of institu 
tional factors, including scheduling, task description, and 
administrative expectations, militate against a more collegial 
working relationship with respect to the teaching role. Tlie 
isolation prompted by these institutional constraints often 
is exacerbated by teachers themselves who, for whatever rea 
sons, feel uncomfortable opening up their classrooms to or 
frankly discussing issues about teaching with colleagues. Tlie 
reiLSt)ns for ibis resistance are varied. Some faculty insist that 



32 



classrooms are their prh^ate domain. Others find, as Menges 
suggests, disparities in expertise, experience, or status 
between themselves and colleagues intimidating or threat- 
ening ( 1987). Others, though, may be uncomfortable working 
in formiil relationships with professional staff developers, pref- 
erring instead "a close and informal relationship with a peer" 
(Menges i987, p. 83; see also Braskamp 1978, pp. 2-3). 

Differences in status can be overcome, however, if there 
is candor in acknowledging problem? and in acting on the 
need to improve. In one successful instance, 

a senior professor and department chair discovered, much 
to his chagrin, that student evaluations indicated room for 
substantial improvement in his teaching At the next depart- 
ment meeting he put his concerns frankly before his col- 
leagues and asked for their assistance in identifying what 
and hoirhe could improve. He shared teaching materials, 
discussed his current teaching methods, and invited his col- 
leagues to observe him in the classroom. Once his colleagues 
understood the seriousness of his request, they did as he 
asked, visiting his classroom, talking to his students, review- 
ing both the structure and the organization of his courses 
The entire group shared and discussed together the teaching 
evaluations of each member. Among other things, this pro- 
cess enabled the chair to change his instruction in ways that 
resulted in substantially improved teaching and better 
course evaluations {9ev^ 1992, p. 5A). 

This situation probably is exceptional but demonstrates that 
faculty can work together to improve teaching if a commit- 
ment to do so exists. A reluctance to work collaboratively in 
this way contrasts markedly with prevailing praaice regarding 
research, where it is quite common for faculty to seek advice 
from colleagues before submitting manuscripts for possible 
publication. It also is becoming increasingly common for 
faculty to work collaboratively on research projects that 
eventually lead to joint publication of the findings (Austin 
and Baldwin 1991). 

An anecdotal record of a tenured, senior linguistics pro- 
fessor who collaborated with a colleague also is illustrative 
of what can he accomplished when teachers work together 
to improve teaching. In his commentary, Carton writes: 
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The examination of my classroom allowed me to reflect 
upon the degree of virtuosity and range of skills in cUiss- 
room management I had acquired ot^er the years. Whether 
my skills are, from a public point of view, praiseworthy is 
a question I had received little information about duritig 
those years in which I had hammered out some skills in 
almost total isolation from my peers who should perhaps 
have been more interested. Nor have I had the opportunity^ 
to watch enough classes of others to be able to develop some 
yardsticks by^ which to make judgments about the quality 
of my own classroom skills. At last, [my colleague] had led 
me to identify my skills arjd begin to make an inventoryf 
of the techniques ! had at my disposal ( 1988, p. 58). 

A finding by Baldwin may go even further, suggesting that 
collaboration is an indicator of vitality among faculty (1990). 
He notes that a substantially larger proportion of 'Vital" pro- 
fessors rather than a less capable "representative" faculty 
stated that they sometimes collaborate with colleagues, t;ike 
professional risks, and engage in innovative or nontraditional 
professional activities. 

Despite the complexity of and probable interactive effects 
between the academy s constraints and the resistance of 
faculty, it still is somewhat surprising that so few faculty con- 
sult with each other on teaching since, as McDaniel observes, 
"the most widely accepted principles of adult learning suggest 
that adults thrive on collaborative learning. Adults are mot- 
ivated by peer involvement and support, and sharing their 
experiences is a powerful resource for learning" (1987, p. 94). 
If collaboration in teaching is to become normative rather 
than exceptional, ways must be found to promote such 
collaboration. 

Rationale for Comprehensive Faculty Evaluation 

Peer review may be essential if significant improvement in 
certain aspects of teaching is to occur (Centra 1993; Soderberg 
1986). Yet, peer review is only one part of a broader process 
for improving this professional role. Another critical element 
is honest and thoughtful self-evaluation (Cross and Angelo 
1988; Stevens 1988). Data provided by students, experts in 
S[^ccific aspects of teaching and learning, and academic 
administrators also may be considerably valuable. Information 
from various combinations of these sources can be obtained 




in many ways: from student ratings of courses and instructors, 
from inteiviews with students and instmctors, from direct 
classroom obser\'ation, from videotapes and audiotapes of 
classes, from assessment of course materials and instaictors 
evaluations of the academic work of students, and from a 
study of other measures of student achievement (Aubrecht 
1984; Menges 1991 )• The process should include looking at 
the pre interactive (course planning and class preparation) 
and the post interactiw (reflection and revision) phases of 
teaching as well as actual delivery' of instaiction (Soderberg 
1986). ' 

Assessment for instructional improvement is only one pur- 
pose for which evaluation takes place. Obviously, summative 
evaluation is another. There ver\' well may be other valid rca- 
st)ns for evaluating C(.)urses and instructors. Faculty evaluation 
including ft)rmati\'e and summative components (and any 
other components deemed appropriate by an academic com 
tnunity ), in which multiple constituencies are consulted and 
several methods and procedures arc used to gather relevant 
data, are components of ihe prcK'ess of comprehensive faculty 
evaluation. 

NearK' all writers in the field of faculty evaluation recom- 
mend the adt)ption of comprehensive programs of faculty' 
evaluatit)n (Aleamoni 1981; Arden 1989; .^rreola 1984; Black- 
burn and Clark 19"^^; Brandenburg. Braskamp, and Or>' 1979; 
Centra 19^9, 1993: Cohen and N4cKcachie 1980; Dressel 1976; 
Grccnwotxl and Ramagli 1980; Romberg 1985; Sauter and 
V;^ilkcr 19-6; Schneider urS; Scriven 1980, 1983, 1985; Seldin 
1984; Smith et a!. 1988; ,Soderberg, 198S, 1986; Spaights and 
Bridges 1986; Stevens and Aleamoni 1985; Study Group 1984; 
Swanson and Sisson 1971; Wolansky 1976). In insisting that 
it is critical to ha\ e a comprehensive program of evaluation, 
as opposed io a more limited one. in place, Dressel observes 
that "no one method by itself is adequate. In fact, overem- 
phasis on one method may do more harm than good. Various 
facets of the program can be examined by different and 
apprt)priate means of assessment" ( 1976. p. 338). 

Batista obser\-es that "no technique or source of informa- 
tion is valid per se in evaluating college teaching. Usefulness 
depends on the objecti\es to be reached" ( 1976, p. 269). Like- 
wise, C^reenwood :ind Kamagli conclude that: 

Ao;/e of the means of craliuitin^ college leaching tised alone 
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seetns to hare a research base which indicates that it is a 
sufficiently valid measure of the teaching effectiveness of 
a given professor. Such a situation suggests the development 
of multiple data systems that are continuously validated 
and subject to ongoing empirical examination of the 
interrelationships existing between the different kinds of 
evaluation and instructional improvement data collected 
(1980, p. 681). 

However, it is not always possible for colleges and universities 
to commit the necessary resources to implement all aspects 
of a comprehensive program of faculty evaluation (Branden- 
burg, Braskamp, and Ory 1979). In this event, scholars usually 
recommend that the university- community ask— and answer, 
after careful deliberation— three fundamental questions 
before deciding what elements to implement, lliese questions 
are: For what puq^cxse is the evaluation being conducted? 
From what sources will the information be obtained and/or 
who will interpret the information gathered? What methods 
and prcx:edures will be used to gather the information? (Bui- 
c(Kk 1984; Cancelli 1987; Craig, Redfield, and Galluzzo 1986; 
Licata 1986; Millman 1981; Prater 1983; Scriven 1980). Further, 
they almost invariably emphasize that a thoughtful cinswev 
to the first of these questions should precede attempts to 
ansv^'er the others. 

A list of questions posed by Fittman and Slate gets at some 
of these same issues, but has a somewhat different f(KUS 
( 1989). Their questions are: Wliat is the overall aim of the 
evaluation? Wliat areas of faculty responsibility are to be eval- 
uated? >X^at are the objective limits of this evaluation? How 
does one establish a framewx:)rk for evaluating thiS/ these 
aim(s)? These authors say that "to omit such questions and 
their consideration is likely to produce a faculty review pRX'e- 
dure with a weak or nonexistent conceptual base. Such a sys- 
tem creates rather than ( resolves] problems" (p. 39). 

Ans'U'ers to questions like these should be sought because 
information from some sources and from some methods and 
prtKedures mny be relevant for one purpose^ but irrelevant, 
or less relevant, for another. To illustrate, several scholars (Hul 
C(Kk 1984; Centra 1975; Cohen and McK'-achie 1980; ScTiven 
1980. 1983; Sorcinelli 1984; Stodolsky l'^ i t) argue that direct 
classroom observation for summative evaluation, whether car 
ricd out by colleagues or by academic administrators, is not 
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normally appropriate or valid. That argument is based on low 
interrater reliability' of observers' findings, observ'ers failure 
to make enough visits to obtain a representative sample of 
teaching behaviors, and the likelihood that the classroom 
teaching and learning environment is different when 
obser\'ers are present than when only students and teacher 
are in the room. In one experimental study, for example, 
researchers found that professors are much more likely to 
im'olve students in the teaching and learning process when 
they know observers are present than when they are unaware 
of the ohsen'ers* presence (Ward, Clark, and Harrison 1981). 

On the other hand, scholars see great potential value for 
classroom observation when it is used for formati\^e e\^alua- 
tion. That argument is made well by Stodolsk\': 

/;/ fonyiatire evaluation, direct observation may be ven' 
appropriate if too much is not made of any given obser- 
vation. Direct observation can provide useful occasions for 
dialogue with superiors and colleagues. Specific occasions 
are what teachifig is all about, and may provide a very 
appropriate focus for discussifig improvement. Discussions 
and suggestions that follow obsetvations of a teacher may 
even be more helpful if it as recognized that he or she might 
teach differently in different situations ( 1984, p. 17). 

To illustrate again, Edwards ( 1974) and McKeachie ( 1986) 
argue that academic administrators should play a significant 
role in summative evaluation, where it is likely they have rele- 
vant information about all areas of professional performance, 
hut only a supportive role in formatiN'e evaluation. Tliese writ 
crs reason that faculty are likely to feci threatened hy having 
administrators involved in formative evaluation, fearing that 
weaknesses identified in this prcx^ess will be used against 
them later when personnel decisions are made. In formati\'e 
e\'aluation, administrators may be better suited for behind 
the scenes roles as producers— providers of time and other 
resources and a generally supportive atmosphere— than as 
actors. 

To be comprehensive, faculty' evaluation must involve peer, 
student, and administrator evaluation and selfcissessment. 
Not all of these cxMistitucncies figure eciually for all evaluation 
purposes, however. Some of them may be required for all of 
these purix)ses, but others may be suitable only, or mostly. 
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for either formative or summative evaluation, or for ;orne 
other purpose. 

Comprehensive Evaluation Models 

A number of conceptual models of comprehensive faculty 
evaluation have been developed, and every once in awhile 
a relatively comprehensive program has been put into place 
at a college or universit\'. At this point, one of the conceptual 
models and a program that actually was implemented are de- 
scribed. After that, a program of formative evaluation of the 
teaching role only, based on a theory of evaluation and pilot 
tested in elementary schools, is described because it includes 
many of the elements subsequently employed at colleges and 
universities for evaluation of this type. 

We present these models here because we are convinced 
that instaictional improvement is contingent on information 
about several interdependent teaching prcxvsses, requiring 
input from several of the academy's constituencies. We don't 
want readers to think that we are under the illusion that for- 
mative peer evaluation is a 'Tlx all" approach. Rather, we 
believe that the models presented ^s'ill illustrate how impor 
tant a comprehensive program is in impro\'ing teaching. 

Soderberg model 

The instaictional evaluation model dc\*eloped by Soderberg 
( 1986) is three-dimensional. At eveiy point, elements intersect 
so that it is virtually impossible to miss any teaching prcX'ess, 
time phase in which an instructional event occurs, or source 
of information for assessing what has taken place. Figure 1 
shows how the model's three dimensions are related. 

The first dimension consists of a series of interdependent 
processes: goals and objectives (asking and answering such 
questions as "What are we trying to do?" and "Wliat are our 
purposes at this point?"), methods and materials ("How will 
we go about doing what we decide to do?" and "Wliat can 
we use to accomplish our goals and objectives?"), and feed- 
back ("How will we go about finding out how we're doing?" 
and "How can we assess the relationship between our goals 
and objectives and our methods and materials?") (pp. lS-16), 
Ans*Vv'ers to these questions may depend on the institution's 
concept of what an educated person should be, on the insti- 
tution's mission, and on individual teachers' epistemological 
values. 




FIGURE 1 

Soderberg's three-dimensional model of faculty evaluation 
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learning styles. Theynvere quite verbal a?id reflective. They 
were able to demonstrate how they consistently took indi- 
vidual student needs into consideration in their planning, 
monitoring and adjustments during their lessons. It was 
evident that they knew their students well. In addition, dur 
ing the lesson, these teachers provided great clarity^ with lots 
of cues, structuring comments, advanced organizers, and 
the big picture for their students They seemed determined 
and strong-willed about accomplishing their objectives and 
having students succeed. 

Students were challenged and at the same time cooper- 
ative and mutually supportive of one another. Above all, 
these teachers were risk takers, willing to make necessary^ 
rei.ision^ and try> neiv teaching strategies Being open, taking 
chances, making immediate changes, and experimenting 
with new models of learn ir^g and teaching are qualities that 
appeared to make these teachers inspiring to both their stu- 
dents and the external c^/'«///^?ror5(Chenoweth 199 L p. 



The third dimension of Soderberg s model is represented by 
the constituencies that are in a position to provide information 
about faculty strengths and weaknesses. Students and 
informed peers are seen as primary sources; informed admin- 
istrators, alumni, and other sources (including self-assess 
ment) are seen as secondary' sources. Students are in a fav(;r- 
able position to evaluate g(xds and objectives, methods and 
materials, and feedback when these processes rekue to inter- 
action. Peers are» according to Soderberg, best equipped to 
avSsess processes related to pre interaction and revision. Tiie 
Soderberg model warrants careful study. 

Romberg model 

Romberg describes prcx'edures empk)yed in summative and 
formative evaluation of programs a k1 personnel at the dental 
school of the University of Maryland ( 198S). in this program, 
several methods of evaluation are employed and sources con- 
sulted in five broad areas: student evaluation of instruction, 
including both course quality and teacher cfTectiveness; eval 
uation of the faculty for decisions regarding reappoint mem, 
promotion, and tenure; the evaluation of c ich department, 
including the formative evaluation of its gcxils and objectives 
by the faculty as well as administrators, and the formative eral- 
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uation of the faculty by peers and department chairpersons; 
faculty and student evaluation of administrators, including 
the dean, associate deans, and department chairpersons; and 
evaluation of the goals and objectives of the school by stu- 
dents, faculty', and administrators. The procedures include 
e\'aluation of course materials and instructors' evaluations of 
the academic work of students but not direct classroom obser- 
vation or class videotaping. 

The processes described in two of these five areas are quite 
conventional in that students at most colleges and universities 
are afforded opportunities to evaluate courses and instruaors, 
and committees composed of faculty and administrators at 
most colleges and universities are asked to make recommen- 
dations concerning personnel decisions. Systematic evaluation 
in the remaining three areas is much less commonplace in 
higher education. Tlie mechanisms that have been put into 
place to evaluate departmental goals and objectives and those 
of the school, as well as the formative evaluation of the faculty 
and administration by the faculty', are not employed often in 
colleges and universities. 

Roper, Dealy and Dombuscb model 

The formative e\'aluation of teaching model of Roper, Deal, 
and Dombusch (1976) is based on the theory of evaluation 
of Dornbusch ( 1975). The model under consideration here, 
which was pilot tested in elementary schools, includes seven 
interdependent stages: identifying the participants and deter- 
mining which of them will work together; defining and clar- 
ifying the teaching and learning objectives of each participant; 
setting the criteria by which the performance of each partic- 
ipant will be evaluated, based on the outlined objectives; 
assessing the quality of the performance of each participant 
through the use of direct classroom observation and often 
other methods; critiquing the strengths and weaknesses of 
the performance of each participant; communicating the 
results of each eraluation through direct interaction between 
evaluator and the faculty member whose performance is being 
assessed; and developing a plan for improvement based on 
self-assessment and student evaluation as well as on the 
results of the peer review process. 

Roper, Deal, and Dornbusch have recommended that the 
membership of each pair of participants be determined by 
the participants, believing that this method is preferable to 
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random selection or arbitrary' assignment. Mutual respect, 
trust, and compatible educational philosophies between par 
ticipants, controlled by self-selection, they argue, outweigh 
advantages offered by other methods of selection. Agreeing 
that participants should determine membership of the work 
ing groups, Heller warns that attempts to impose membership 
could result in groups that "go through the motions" but 
make little legitimate attempt to improve the qualitv of teach 
ing (1989). 

In the Roper, Deal, and Dornbusch model and in many 
other formative evaluation models, participants are encour 
aged to identif^^ the merit of their own objectives (either indi- 
vidually or collectively), to determine the methods by which 
data will be gathered, and to establish criteria by which 
strengths and weaknesses of their performance wiW be eval 
uated. Allowing participants to make these decisions is seen 
as a means for developing faculty ownership of the program 
(Heller 1989). Wliile performance is evaluated largely by 
direct classroom obser\'ation, a number of methods for gath- 
ering information— including assessment of course materials, 
student evaluations of their teacher's etTectiwncss, and self 
evaluation— alst) are important elements of the model. 

After the agn^ed-upon methods of evaluation have been 
completed, the group members meet to discuss the findings. 
At these feedback sessions, both participants are encouraged 
to provide an assessment of the items under consideration. 
Participants have ohsei-ved that criticism most often is pres 
ented as suggestions for alternati\'e techniques, rather than 
as mandates or absolutes. Roper, Deal, and Oornbusch note 
that comments "encompassed virtually eveiy aspect of chiss 
room activity. Teachers learned not only about their own per 
formance but about the overall climate of their classrooms" 
(p. 62). 

Planning programs for improvement evolve from the feed 
back sessions, from student evaluation, and from self 
reflection on performance in general. At these improwment 
planning sessions, participants work collaboratively to deter 
mine the strategies that might be employed in efforts to 
improve performance, the resources that may be rec|uired to 
accomplish these objectives, antl the means by which these 
instructional-improvement etforts will he evaluated. 

Roper, Deal, and Dornbusch report that participants wt-re 
enthusiastic about the prt)gram and that many of them 
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planned to continue to work with colleagues in programs like 
this one in the future. There is no indication, though, that the 
program has been widely adopted by elementary or secondary 
schools or by colleges and universities. However, elements 
of it have been used in the isolated formative peer-evaluation 
programs that have been identified. In later sections of this 
report (Incentives and Disincentives), ways that may make 
programs of this t\pe more appealing to faculty and that may 
discourage them from participating, respectively, will be 
addressed. 
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It is difficult to quarrel with Batista's conclusion that "col- 
league evaluation is better used when the categories are those 
in which faculty members are in the better position to cast 
judgments than anyone else" (1976, pp. 269-70). The same 
might be said of students, teaching consultants, administrators, 
and others who supply information about teaching perfor- 
mance. It also should be emphasized again that assessments 
of teaching by any of these sources "are not necessarily valid 
indicators of effective teaching when used by themselves, but 
they may be helpful when used in conjunction with other evi- 
dence" (Cohen and McKeachie 1980, p. 147). 

The conceptual model of Soderberg is instructive in iden- 
tifying areas in which peer review, as well as evaluation by 
other constituencies, are most appropriate (1986). In this 
model, teaching processes (setting of goals and objectives, 
determining what methods and materials will be used in 
teaching, and providing feedback) interact with the time 
phases (pre- interaction, interaction, and revision) at which 
instructional decisions and actions occur and with the con- 
stituencies (faculty, students, academic administrators, and 
others) that could provide relevant information about teacher 
performance. 

Soderberg suggests that faculty colleagues are best qualified 
to assess processes occun-ing at the pre-interactive and revi- 
sion phases (1986). Cohen and McKeachie (1980) and Seldin 
(1984) envision an even broader role for colleagues, includ- 
ing assessment of what cxxurs during deliveiy of instruction 
(and when students are otherv\'ise engaged in learning) as 
well as what cKcurs before and after. Hart advocates a par- 
ticularly prominent role for faculty as classroom observers, 
noting that faculty have expertise about teaching and learning 
that students simply aren't in a p(Xsition to possess ( 1987). 
Katz and Henry also recommend classroom observation by 
colleagues and suggest that it be combined with interviews 
of selected students (1988). 

In this section, we examine a range of roles that scholars 
suggest as appropriate for faculty to play in formatix'e evalua 
tion. We cite primarily the writings of Batista ( 1976), Cohen 
and McKeachie ( 1980), Hart ( 1987), ScTiven ( 1985, 1987), 
and Soderberg ( 1086) but refer also to the scholarship of 
tether writers on specific aspects c^f evaluation. We also look 
briefly at the potential and limitations of students, teaching 
consultants, and academic administrators in the process of 
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teaching improvement. 



Faculty Roles 

Cohen and Mckeachie provide a useful classification of roles 
that faculty could play in evaluating colleagues' teaching 
( 1980). The categories of this classification are: elements of 
course design, including goals, co^^rse content, and organ- 
ization; methods and materials employed in delivery of 
instruction; evaluation of students and grading practices; and 
integration and interpretation of information gathered from 
students, administrators, and self-evaluation as well as eva- 
luation by peers. 

There is considerable overlap between this classification 
and Scriven's listing of criteria on which teaching evaluation 
should be based (1985). These criteria are: 

The quality of the content taught (does the teacher really 
know the subject ivell enough to provide sound and illum- 
inating answers to any questions that the best student could 
legitimately ask about the actual or required curriculum 
content); the success in imparting and/or inspiring learning 
(which includes learning the value oj learning oj systematic 
inquiry, cooperation, etc— that is, learning is not restricted 
to the cognitive domain); the mastery of professional skills 
(how to set valid tests, deal with the nonclassroom duties, 
etc. ); and the adherence to ethical standards (aividing 
racism and favoritism, etc ) (p. 36). 

Elaborating on these criteria, Scriven develops especially the 
second one, noting that faculty' have responsibility in "increas- 
ing the amount of valuable learning acquired by the students, 
. . . increasing their capacity for learning" ( 1987, p. 10), and 
"increasing [their] learning to something like the level at 
which the students are capable" (p. 21). 

Scriven does not address the issue of hov/ success (or fail- 
ure) in meeting these criteria are to ^ e determined, except 
for insisting that specialists take a loc.lc at tests written by 
faculty and the grading of these tests ( 1985, 1987). Fellow 
faculty- may be in a particularly favorable position to evaluate 
iispects of all of these criteria, as Cohen and McKeachie sug- 
gest ( 1980). 



30 

■45 



Elements of course design MrKeachie sue- 

m the category of course design, Cohen 

gest that faculty colleagues examine the following. 

. The professors mastery of course content. 

. Annrooriateness of course objectives. 

. SeSroTof course content (knowledge of what must 

be taught). 

• Organization of the course. 

• Coverage of appropriate content. 

. incorporation of recent scholarship into selection of 

. SuSity of student a.ssignments in meeting course 
objectives. 

In this same general area, Soderberg suggests that informed 
in its contextual environment (1986). 

In^MSeaS cite Rve areas in which colleagues could 
a^iss the effectiveness of fellow faculty: suitability of 
"Sods of instruction in meeting course goals appropn^ 

«ic« otlppropri-aK me.hodoloe.es fo. .eaehing speafe con- 
tent (1976). 

^uSn Sgnments and their grading practices. Cohen and 
McKeachie fecommend that colleagues examine: 

P' ---^ - examinations and on other 
assignments. 
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* ImT-k"''' ^^■^"■■•/'-^^l^'irecl of .stfclents to complete writ- 
ten assignments and other projects 

* .o";ud;nts"" ' ^"''"^ '''''''' -1^'^--' 

^^r^^Zl^^'^f' ^-on^Pctencies in this 

St mm n , to course ob,ecEi^•es and prcK'edures. the asea,!- 
Kss o the evaluation to students in the learning process and 
.lie relaconship bet^^ een awarding of course gn^ls nd J 
8.ad.ng system communicated to students ( 1986) 

som en emplKisizes the importance of t^vc) related items- 
est constructK,n and the grading of student examinations 

--^ -ca.:ure. 

men s assess he competence of colleagues in these areas 
he ol^ser^■es that "few teachers, from kindergarten to pe s " 
g.aduate, have e^•er had their tests and scoring kevs h c ked 

l av 1 " 'rVT" ''''''''' ■'^•^'"^^'■^■'^^ 

1 1?85: ; S) ' " ' p'^---" 

The professional competence to vs-hich Scriven refers is ■ 
Kla ed to recommendations that experts check to see that 
professors grade essay tests.- -blind. ' so that biases - n 
a«a,nst .students are n.inin.ized: question by c,uestion. S er 
han test by test, so as to reduce potential -halo effects- -.nd 
7^^-"'^'"y"ito.m .standards of evaluation from the fir 

; ■ Z" T " '''' ^^"^'^^"^ '^^'J^'- ^ '■•slH.ffling- the 

pa e,s after reading an.s.sers to each c|uestion ) so that re-.der 
en huMasm. frustration, and fatigue afilvt students in no 

vu it i'sTrl • '"""'"""^^ "^ade 

; -"';'«-H.s to know the name a.s.scK iated with ,he 
Noik. in th s .scenario, the instructor can bring to the evahn 
•■on an understanding <,f the stttdenfs prior k' owledge an 

HKiue backgrcnind and perspectives. Ihis mav shed ligl 
upon a student s response that tiiight be lost in blind re^•ie^^- 

h IptuI to colleagues ,n the construction of -objective" t-st's 
where s\r.ting good c|uestions often pnnes difTicult. ' ' 

Integration ami iutetpretation 

Cohen and McKcachic itidic-ate that colleague.s are idc>alK- 
^■•imppecl to integrate and inteiprct information gathered' from 
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various methods of evaluation and from all sources providing 
information. In the evaluation of instruction, they suggest that 
peers assess the following: student ratings in light of circum- 
stances under which the course was taught (e.g., large vs. 
small enrollment in classes, required vs, elective course, and 
a number of other contingencies that can affect student rat- 
ings); the criteria used in evaluation instruction; and the 
weighing of the criteria used in determining teaching 
effectiveness. 

Batista lists a number of other factors that could be con- 
sidered by colleagues in the integrative stage of instaictional 
evaluation: 

1. Faculty members' ov^n evaluations of their teaching. 

2. Faculty members' own evaluation of their knowledge of 
specific content areas within the field as a whole. 

3. Informal course evaluations conducted by instructors with 
their students. 

4. Alumni ratings of faculty members. 

5. Student achievement in courses. 

6. Interviews with groups of students or individual students 
(1976). 

Craig, Redfield, and Galluzzo (1986) and McKeachie (1986) 
recommend that the integrative process also include a study 
of the relationship between students' explanations of their 
responsibility in leaching and learning and their evaluations 
of courses and instmctors. These researchers are convinced 
that information provided by such comparisons would be a 
valuable tool in assessing both quality of teaching and student 
learning. 

While integrating and interpreting information appear to 
be important, they must not he considered faculty's only 
responsibility. If faculty are not willing to examine aspects 
of teaching they are uniquely qualified to assess, there is that 
danger that: 

What is peer reviewed' not the process of teaching and 
its products (the learning that the teaching enabled) hid 
fnierelyj the ohsen^ations and ratings submitted by students 
and assorted others. This situation is in part a function of 
faculty^ uneasiness about the instruction of colleagues in 
their cUissrooms But underneath this uneasiness lies a more 
troubling circumstance: the lack of clarity* about why faculty 
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should he ohscrtvrs of one cuiothers teaching [emphasis 
theini (Edgcrton. Hatchings, and Quinlan 1991, p. 5). 

ClearK-. these scliolars belie\-e tliat faculty must become 
in\x)lvcd in assessing se\ eral aspects of colleagues' teaching, 
and that this imolvement should extend beyond an integrative 
function. 

Peer observation 

>X1iiIe Cohen and McKeachie acknowledge a need for forma- 
tive peer obserxation. their classification deals with this 
method only implicitly. Other scholars provide more detail 
on how peer obserx-ation could be employed. 

Mart, believing that faculty should play a broad role in 
instructional ImjM-ovement, identifies events that occur during 
deliverv' that faculty are especial!}' well-qualified to assess 
i 19K^). These events are: the place where and the time when 
classes are taught and other physical factors affecting deliveiy 
ot instruction; the procedures used by the teachers in con- 
ducting the class; the teacher's use of language to inform, 
explain, persuade, and motivate, and the language students 
use in responding and reacting to the teacher; the roles played 
by teachers and students as they interact; the relationship of 
what is occurring in a particular class to other courses, dis- 
ciplines, and the curriculum in general; and the outcomes 
of teaching, as reflected in student learning. These roles are 
discussed in more detail in the next section of the report, 
where the place of direct classroom obserx-ation is si:)ecifically 
addressed. 

Recognizing that faculty have the potential to contribute 
to teaching improvement of colleagues in several significant 
ways. Seldin ( 1984) filLs in and expands on areas in which 
other scholars ha\e agreed that faculty have expertise. The 
following cjuestions framed by Seldin could be used by faculty 
as the\' assess colleagues' performance. 

W'e belie\'C that scholars have demonstrated, rather con 
vincingly, thai competent faculty can critique colleagues' 
teaching and assist their peers in imi:)roving that teaching. Sev- 
eral methods are available for doing so. 



In the remainder ot this section, we look briefly at some of 
the strengths and limitations of students, academic admin 
istraiors. and leaching consultants as j^-oviders of information 



in formutix'c evaluation of teaching. Wc make no attempt to 
be comprehensive but liope to sliow instead that eacli con- 
stituency can contrilxite relevant data, though none of them 
can provide all the information needed by professors who 
wish to improve their teaching. 

Students 

Students evaluate college teachers and courses more than an\' 
other constituency (Bergman 1980; Seldin 1984). Tliey are 
relied upon not onK' because it is relatively eas\' to devise 
teacher-rating forms, administer the forms, and tabulate tiie 
results (Abrami 198S) but also because, as Cro.ss and Angelo 
.say, '\students ha\e ample opportunity to see teachers in 
action in good days and bad. [and] they are in a good posi 
lion to evaluate the Impact of the teacliing on ihem.selves as 
learners * ( 1988. p. 12S). It is difficult to deny that students 
are in a position to provide reliable information about certain 
t\pes of teacher behaviors. 

Students probai^ly are mcxst qualified to evaluate a.spects 
of teacher performance occurring at the deliver}' phase ot tiie 
instructional process (Soderberg 1986). Tliey .seem le.ss ciual- 
ified— even unciualified—to a.s.se.ss many aspects of perfor- 
mance occurring at the pre- interactive and revision phases, 
except for how deliveiy is impacted b\' e\'ents occurring 
at other times. 

NXlien completing teacher riitii^g forms, students usualK' 
are asked to a.s.se.ss several elements: .style characteristic.-* as.so- 
ciated with del i\'eiy (class preparation, organization, .sen.se 
of humor, entiuisiasm. rapport with .students, etc.. called 
"oKsciA-ables** by Bulcock [ 198^1 and "secondaiy indicators** 
by Scriven 1 198*^ | ); their perceptions of what they have 
learned and of the in.siructors command of the subject matter; 
and overLill impressions of the quality of the in.struclor's teach- 
ing and the cour.se. Some of the information they provide 
probably is nece.s.saiy for .summaliw evaluation (for example. 
gk)bal ratings of the teacher and cour.se). Other information 
may be useful in formative e\Liluation. Some of it may be 
invalid for either purpose. Students, like ever\' other coasti 
tuency. have .strengths and limitations in the faculty evLiluation 
pre )ccss. 

To determine what .students are. and are not. in a favorable 
po.siiion to evaluate, it may be instructive to look again at the 
criteria on which facult\- evaluation should be based. Accord 
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ing to Scriven, there are hut four such criteria: quahty of con- 
tent taught, the instructor s success in teaching that content 
and in inspiring learning, the instructors masten- of profes 
sionai skills in writing tests and evaluating the academic work 
of students, and the in,structor\s adherence to ethical standards 
( 198S ). Students appear qualified, as c)b,ser\'ers of classroom — 
and certain out-of-ckLss— proceeding.s, to a.ssess the in.struc 
tor's success in organizing and delivering the course content 
and in.spiring learning, and the in,structor\s adherence to eth- 
ical .standards a.sscK'iated with teaching, although other con 
stituencies also are in a favorable position to comment on 
these factors. Except in the mcxst egregious instances, where 
faculty are clearly incompetent in knowledge of subject mat 
ter, for example, faculty and academic administrators are in 
a considerahk- better position than .students to evaluate the 
quality of the content taught and, with the help of experts 
in tests and measurements, the te.sts written by teachers and 
the quality of the academic work submitted by students. Stu 
dents too often are asked to evaluate much that lies outside 
of their areas ofexperti.se. 

Hxce[:)t for .some po.ssible connection to "succe.ss in impart 
ing and or in.spiring learning," the .style characteristics that 
teachers exhibit as they deliver in.struction are not related 
directK to Scriv en s criteria ( 1985). And even with that dimen- 
sion, there ma\' be better indicators than the style character- 
istics. \cl as Scriven suggests, such ^'secondan* indicators" 
ma\' have; 

{f Ksc/ifl role In Jorwatitv et'aluatio)i cisfoHoirs. If yon hare 
(ie})H»i^tratcd that a teacher is dot tig hacify, t/siug the [proper 
criteria, then the "anthology of successful styles'' bid/t up 
by researchers prorides a txiluahle resotirce for su^estions 
as to practices the teacher tuight consider adding to his or 
her cunxnit repertoire in the quest for improtvmoit ( 1987, 

The validity of student ratings may be called into question 
in a number of other ways, as Abrami ( 1985) and others have 
suggested, when: 

1. 'leachcr rating forms include a disproportionately larger 
number of items on '\succe.ss in imparting and or iaspir 
ing learning/' as Scriven ( 198"") phrases it, and a dispro 
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portionaicly smaller number of items on other valid eval- 
uation criteria; 

2. The forms include greater numbers of items on interaction 
between students and instructor and fe%ver on important 
aspects of teaching that occur during pre interaction'and 
feedback phases of the instructional process; 

3. Forms include items appropriate for some classes, aca- 
demic disciplines, styles of teaching, and teaching meth 
odologies but less appropriate (or inappropriate) for oth 
ers; for example, the forms may be better suited to the 
social sciences than the humanities; and 

A student's response to one item affects responses to other 
items, a threat to validity called the "halo effect"; this 
effect may bias results either for or against a teacher, 

Abrami also notes that: 

Stinlents CIS a i^roup tuay he hiatxumtc ohsvnvrsfor a rci 
rieiy of rccisons: they ffiay he uaire and insetisitire to qual 
itative differences in ifistruction: they may be collectirely 
biased by their own expectations which distort their percep- 
tions: landl they may he unfairly lenient hi judging teach 
1)1^ effectiveness { 198S. p. ID. 

While information provided by students \ia teacher rating 
forms can contribute to an understanding of teacher perfor- 
mance, the picture they paint often is incomplete. More detail 
can be filled in when this information is augmented by data 
gatiiered from other sources, including the faculty. 

Academic administrators 

Because academic administrators usually were facult\' mem 
bers before assuming their present positions, or continue to 
teacii as administrators, they, like faculty, have experti.se about 
teaching and learning, teacher performance, concern about 
colleagues teaching, etc., that .students don't have. They are 
in a good position to evaluate aspects of teaching that occur 
at the pre interactive and feedback phases of in.struction as 
well as when instruction actually takes place. Like faculty and 
teaching consultants, they should be able to assess quality 
of content taught, the instructor's .success in as.se.ssing .student 
work and in inspiring learning, and the procedures employed 
in leaching content. Hut admini.strators are further removed 
from the faculty than their colleagues, 
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Since much of an administrator's time is consumed by 
responsibilities other than teaching, administrators are less 
likely to interact with faculty to the same degree and in the 
same ways as fellow faculty. There also is the likelihood that 
faculty will be reluctant ro seek help from administrators in 
improving their teaching, believing that deficienci<'s in per- 
formance will be used against them when personnel decisions 
are made, lliese faaors may limit the effectiveness of admin- 
istrators in a process aimed at instructional improvement. 

Teaching consultants and faculty development 
programs 

Teaching consultants normally offer assistance to faculty in 
one or more of the following broad areas: instructional devel- 
opment, personal development, and institutional development 
(Bergquist and Phillips 1975). The type of programming 
offered depends in part on the philosophies of the consultant 
and of the institution's administrators, and on time and other 
available resources. Interest in faculty development has waxed 
and waned from the time the first programs were put in place 
in the early 1970s, but considerable interest in them has been 
sho\^^l in recent years. 

Teaching consultants usually have expertise in a number 
of teaching areas: strategies, student learning, learning styles, 
and technology. There is little doubt the>' can be catalysts for 
improved teaching among faculty who are motivated to work 
with them. 

Mathis, in obser\'ing that many faculty development officers 
are psychologistvS by training, concludes that: 

Those who organize institutional programs for instructional 
development should he aware of the 'culture' of the many 
disciplines in higher education. White psychology' may have 
much to say about teaching and learning psychologists are 
not always able to communicate this to their colleagues out- 
side of ps\'chology in a language easily accepted or under- 
stood. The value of having faculty in the many fields of 
study who know the research literature on teaching and 
learning and who can communicate with their colleagues 
in the language of their discipline, suggests that instructional 
development can best be sensed by preparing faculty to per- 
form an instructional development function in their own 
field rather than anticipating salvation from a central hive 
populated with psychologisLs ( 1974, pp. 10- 1 1 ). 
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This admonition suggests that there should be a close con- 
nection between the instructional improvement side of faculty 
de 'elopment and formative peer evaluation. 

Despite being written about 20 yepr. ago, Mathis's advice 
regarding faculty' development centcr> wcms as relevant now 
as it probably was then: 

The Center approach is successful only to the degree that 
Center programs and staff are responsive to the diverse, and 
sometimes conflicting needs of the campiis. Centers should 
not staff themselves to reflect any one orthodoxy about 
teaching The successful Center should be able to assist the 
faculty member who is looking for a teaching system to the 
same degree that it can help a faculty member with sym- 
pathetic advice [Such Centers] ought to avoid academic 
et migelism as much as possible. The temptation to save the 
natives from themselves through an aggressive program of 
prophylaxis, usually technological in nature, is generally 
nonproductive, since it involves programs for the few at the 
expense of the many The natives should save themselves, 
and Centers should be as eclectic as possible in helping them 
doso{\91A. p. 25). 

Although ultimately sustained by a personal desire to succeed, 
buoyed no doubt by mastery of and passion for a field of 
study, good teaching is most likely to occur in a culture where 
teaching is valued and where there is a support system in 
place that encourages its development. The commitment of 
administrators is essential. While the support of all segments 
of the academic community is important, the faculty hav(^ a 
key role because they can look at what takes place in class- 
rooms and beyond from different perspectives than students, 
in more detail than administrators, and perhaps in ways that 
command more confidence and trust from faculty than either 
teaching consultants or administrators. Colleagues, using a 
variety of methcxis, can look at what their peers do and why 
they do it within context of their academic disciplines. 

In the next section, we look at five of these methods: direct 
classroom observation, videotaping of classes, assessment of 
course materials, evaluation of instaictor evaluations of the 
academic work of students, and analysis of teaching portfolios. 
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METHODS OF FORMATIVE EVALUATION 



It seems dear that faculty are well-qualified to assess many 
aspects of colleagues' teaching and related professional activ- 
ities and some of the effects of that teaching on student learn- 
ing. Because faculty expertise about teaching and student 
learning, and their knowledge of colleagues performance, 
are acquired in several ways, a number of methods of eval- 
uation, used in combination rather than independently, are 
helpful to gain insight into an instructor's role in the teaching- 
leaming process and to determine how teaching might be 
improved Fellow faculty' c^n look at some of what peers do 
prior to interacting with students (or before students are oth- 
erwise engaged in study), at what occurs when a teacher and 
students interact, and at how a teacher evaluates his or her 
performance with respect to student learning, lliey also can 
examine the complex relationships among the following var- 
iables: goals, objectives, and course planning; methods, mate- 
rials, aiid procedures; and the feedback students receive from 
teachers and the teachers assessment of student learning. 

Five methods that have been used by colleagues to assess 
their peers' teaching for the purpose of instructional improve- 
ment are: direct classroom obser\^ation, videotaping of classes, 
evaluation of course materials, assessment of instaictor eval- 
uations of student academic work, and analysis of teaching 
portfolios. The merits and limitations of the first four of these 
methc/ds, and what can— and cannot— be learned from and 
how to use each of thx^m, are examined in detail in this sec- 
tion. Also examined, though in somewhat briefer form, is the 
role of the teaching portfolio in instructional improvement. 

Direct Classroom Observation 

While most scholars express serious resemtions about the 
use of direct classroom observation in summative evaluation, 
nearly all of them agree that it can be employed effectively 
in formative evaluation. A number of scholars believe that 
peer obser\^ation is essential if the evaluation is for instruc- 
tional improvement. 

Objectives 

Scholars argue that peer obscr\'ation should he employed 
because faculty members ha\'e expertise in the process of 
leaching and learning that is not pcxsscssed to the same degree 
by either students or administrators (Brasfcimp 1978; Centra 
1986. 1993; Cohen and McKeachie 1980; Shulman 1993: 
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Soderbcrg 1986; Sorcinelli 1984). Some of them also suggest 
that it is' a vital component of the paK'ess, becausse it is the 
only way in which some aspects of teaching can he assessed 
adequately. 

hi regard to improving the quaHty of instruction through 
classroom ohser\'ation, Hart observes: 

To iniproiv, teachers tieed the help) and support of other 
teachers. Teachers tieed to consult regularly, orerari 
extended period, uith other teachers. Teachers need to 
ohsenv other teachers at work, he observed by them in 
return, and share their observations, reflections, and recom- 
mcndat ions {19^7, p. \^). 

>Xliilc acknowledging a practical problem asscx:iated with 
classroom obseivation, Weimer, like Hart, stresses some of 
its important beneftts: 

iFacultyl need to be in each other's classes regularly, rou- 
tinely. To expect that to occur may be tiaive and unrealistic. 
Faculty labor under multiple demands. Nevertheless, obser- 
vations . . . should not be special, one time activity. They 
need to be an ongoing part of teaching. They keep instruc- 
tors fresh, encourage and develop accurate self assessment, 
and make obvious the complexities of the teaching- learning 
phefiomefiofi ( 1990, p. 122). 

But it should be emphasized that direct classroom obser\-ation 
may not be "eas\', comfortable, simple, or quick in results" 
(Hart 1987, p. 15). 

Wliile several programs in which peers have observed a 
colleague's classroom have been presented in the literature 
(some of them will be described in the next section), much 
less has been written about the specific events that might be 
observed. A particularly cogent discussion of six inteiTclated 
categories of these events is provided by Hart. 

I. Tlie physical temporal setting. Hart notes that the: 

time of day, room size at ui shape, air ( or the lack of it), 
light ( or dark ), surrounding noise, furnishings, apparatus, 
and clutter /affect/ ' " the people of the event use or mi*; 
use this environment: their uses of space, access, position ing,' 




clLstcuice, mobiiitics. . . . IWhilvj the teacher may }uvd Jeir 
roninciers oj the ecologv, (he reactions of the observer may 
well help to understand and use it better ( pp. P- 1 8 ). 

2. Classroom structure and proccelures. Hart ohscives ihLit 
each class has: 

its intellectual structures, orders, secjuences. its texture (jJ 
goi erning idecis, its proportions, connections, transitiofis, 
planned or not Some teachers regularly signal to the class 
what these are, others ( ill advisedly, I think) tahe them for 
granted. Teaching is, among other things, a co! :^()sing pro- 
cess. . . . The observer can, at least, keep track of the struc 
tures or logics that are connnunicated. and report them 
back—occasionally to the teacher's surprise (p. 18 ). 

3. The rhetorical dimension. Hart notes that certain types 
and levels of language are employed by teachers and .stu 
dents and concludes that: 

they a*'e sonietinws sinu'lar, sometimes quite distinct— even 
separate or divisive. . . . But not many teachers in myexpe 
rience are aware of the languages they use. The observer 
can hear and report the relative degrees of difficult)', for- 
mality, technicalit}\ the dominant syntactic forms . . . liven 
class session has its rhetoric certain fonns and methods that 
are used to achieve certain ends—infontuitive, explanatory 
persuasiiv. . . . To ca^ryout these complex aims, the teacher 
uses certain tactics and methods: cLssignments, exercises, 
demonstrations, examples, analogies, and motivational 
appeals The observer can leani to observe and report the 
ends and the appropriateness of the means ( p. 18 ). 

-I. The dramaturgical .sociopolitical dimension. Mart obser\es: 

W'e are all sufficiently familiar with dramaturgy to be use- 
ful observers of how members of a class play their roles and 
how they interact. \X'e can record siuh phenomefia as pa 
cing I'oicing nonverbal behai'ior afid cofnnuinicatioti, the 
class dynamic its degree of intensity and ifivolvement. 
W'e cafi obsenv how the teacher uses authority ( or power) 
and what kinds, intopret the politics of the cIclks, the direc 
tions and commands, invitations, judgments, rewards, and 
threats {p. 19). 
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5. The curricular context. While the curricular context prob- 
ably cannot be directly obser\'ed, an observer can make 
certain inferences about the class's relationship to: 

largc>r designs, other courses and areas of study, other dis- 
ciplines, lei f els of learning and development, academic goals 
and values, extramural preoccupations and influences. 
No class is an island. What useszre made of such foreign 
relations, and how many, can be observed. What uses and 
how many should, be made is a legitimate issue of strategy 
and priority {p. 19). 

6. Tlie effects of teaching. Hart stresses that the outcomes 
of teaching are what really matter, but notes: 

Most teachers unwittingly cling to the assumption that time 
needed for teaching leaves no time in class for finding out 
what is being learned. . . . The obsen^er cati only tryf to catch 
the clues and report them, and tr}> to help the teacher find 
and use more adequate ways of discovering what has been 
learned {p. 19). 

Hart's classification may seem imposing, even overwhelming, 
to faculty who have not been involved before in classroom 
observation. In program planning, the guidelines might be 
used as a conceptual tool to define what might be accom- 
plished. Later, after classroom observation has been put into 
place, obser\^ers might use the guidelines to focus on specific 
aspects, but not necessarily all aspects, of teaching, 

Peer observation models 

Most programs of classroom obser\*ation in higher education 
are based on a model described by Bergquist and Phillips 
( 1975) or on clinical super\asion models employed in elc 
mentar>' and secondar>' schools. The Bergquist and Phillips 
model has three interdependent stages: contracting, infor- 
mation collection and analysis, and information feedback. 

hi the contracting stage, "the instructor should determine 
what t>pe of inforniation concerning his teaching he wishes 
to rccci\'o'* (p. 88), and he or she and the ()bscr\'cr(s) should 
agree on the prcx-edures to be used in assessing the instaic- 
tor s leaching. These prcK^edures may include, for example, 
videotaping and iiiter\'iewing students as well iis classroom 
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obsen-aiion. Information collcviion and analysis is the stage 
in which obsen/ation and other agreed upon data-gathering 
pnK'edures and systematic data analysis occur. Coliectioii of 
data should he carefulK' orchestrated, witii analysis confined 
to areas of teaching in w hich the course instructor seeks assist 
ance. The information feedback stage usually involves two 
events: a brief written report in which major conclusions are 
outlined and a meeting in which the instructor and ob- 
sen-er(s) discuss findings and recommendations. 

The type of peer obser\'ation proposed by Sorcinelli ( 198-4) 
is like the clinical super\'ision models (e.g., Goldhammer 
1%9) in that the process in\'olves pre c)bser\'ation, obser\a 
lion, and post obserxat ion phases. Tlie questions she suggests 
be asked during each phase (Figures 2, 3. and 4) could be 
particularly useful to faculty with limited experience obser\- 
ing colleagues' classes. Despite being designed expressly for 
peer observation, colleague obser\ ers also must be familiar 
with a peer's course materials, since the observer sometimes 
is asked to compare what is occurring in a colleague's class 
to information contained in his or her course materials. 
Obser\'ers using Sorcinelli's guidelines may need to be cau- 
tioned not to place too much emphasis on teaching strategies 
and deliveiy skills and not to neglect the faculty member s 
content knowledge and ability to communicate it effectively 
to students. 

FIGURE 2 
Pre-Observation Conference Guide 

I. Briefly, what will he happening in the class 1 will ohserxe? 
i.Wliai isyour j^oal for the dass?\Xliai doyou hope students will 
gain from this session 

3. \\1iai do \-ou expect suidenis to do in the class lo reach stated 
j»( )als? 

t. Wliai c an I expect you to do in class?" Wliai role will you take? \\1iai 
teachini^ methods will you use? 

Wliat have students been asked to do to prepare for this class? 
(y Whal was done in earlier classes to lead up to this one? 
^. Will ihis class he generally typical of your leachin.^? If not. what 
will he Llifferent? 

8. Is there anyihui.u specilk on whieh you would like me to Ux i'' 
durin.u this i lass' 

.Vo/i' l-rnn^ .'Nn Approath in C olle.ij^iu' I valuatini^ ot CLissnutm Insmnimii 
In MI) SnrtiiU'lli. \^)H\ JounutI of Instnittional PcrchptncHt " ( O. p 1-4 
Kepnnied h\ jiennissmii 
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nGURE3 



Classroom Observation Guide 



Students* and Teacher^s Attitudes and Behaviors Before Class Begins 

1. Do students arrive noticeably early or late? 

2. Do they talk to each other? 

3. Do the)' prepare for class? Take out books and notebooks? 

4. When does the instructor arrive? 

3. What does he or she do before class (write on board, encourage 
informal discussion with students, sit behind the desk)? 

Teacher's Knowledge of Subject Matter 

1. Does the instructor exhibit knowledge and mastery- of the content? 

2. Is the depth and breadth of material covered appropriate to the 
level of the course and this group of students? 

3. Does the material covered relate to the syllabus and goals of the 
course? 

4. Does the instpjctor present the origin of ideas and concepts? 

5. Does he or she contrast the implications of various theories? 

6. Does he or she emphasize a conceptual grasp of the material? 

7. Does he or she present recent developments in the discipline? 

8. Does he or she present divergent points of view? 

9. Is there too much or not enough material included in the class 
session? 

10. Is the content presented considered imponant within the dis 
ciplinc or within related disciplines?^ 

Teacher's Organization and Presentation Skills 
A Engaging Student Interest 

1. Does the instnictor prepare students for the leaming that is to fol 
low by assessing what they know about the topic through use of anal- 
og)', a thought-provoking question, reference to a common expe 
rience. etc.? 

B. /ntroduction 

1. Does the instructor provide an over\'iew of the class objectives? 

2. Does he or she relate the day's lesson to previous class sessions? 

3. Does he or she use an outline on the board or overhead projector? 

C Organization and Clarity 

1. Is the sequence of covered content logical? 

2. Is the instructor able to present content in a clear and logical 
manner that is made explicit to students? 

3. Does he or she provide transitions from topic to topic, make dis 
tinctions bet\veen major and minor points, and periodically sum 
niarizc the most importiint ideas? 




4. Does he or she define new concepts and terms? 

5. Does he or she use illustrations and examples to clarify difficult 
ideas? 

6. Does he or she use relevai^t examples to explain major points? 

7. Does he or she pro\'ide handouts when appropriate? 

D. Teaching Strategies 

1. Are the instructor's teaching methods appropriate for the goals 
of the class? 

2. Is he or she able to varv' the pattern of instruction through move- 
ment around the class, gestures, voice level, tone, and pace? 

3. Does, or could, he or she use alternative methods such as media, 
discussion, lectures, questions, case studies, etc.? 

4. Is the use of the chalkboard effective? Is the board work legible, 
organized? 

5. If appropriate, does he or she use students' work (xvTiting assign- 
ments, homework assignments, etc )? 

6. Are the various teaching strategies effectively integrated? 

k\ Closure 

\. Does the instructor summarize and integrate major points of the 
class session at the end of the pericxi? 

2. Does he or she relate the class session to upcoming class sessions 
or topics? 

3. Are assignments presented clearly? Hurriedly or drawn out? 

4. Are assignments appropriate to class goals and course level? 

5. Are students attentive until the class session ends? Or are they rest- 
less (talking, closing notebooks, etc.) before the class ends? 

6. What happens after class? Are there informal discussions among 
students or between the instructor and students after class? 

Teacher's Discussion and Questioning Skills 
A Introduction to Discussion 

1. How is discussion initiated? 

2. Are the purposes and guidelines clear to students? 

3. Does the instructor encourage student in\-olvement? 

B. Types of Questions 

1. Are questions rhetorical or real? One at a time or multiple? 

2. Does the instructor use centering questions (to refocus students' 
attention on a particular topic), probing questions (to require stu 
dents to go beyond a superficial or incomplete answer), or redirect 
ing questions (to ask for clarification or agreement from others in 
the class)? 

C Level of Questio^Ls 

I. Wl^iat level of questions docs the instaictor ask? (Lower level ques 
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tions usually have a fixed or "right" answer and require students 
to recall, list, or define principles or facts. Higher level questions 
ask students to generalize, compare, contrast, analyze, or synthesize 
information in meaningful patterns.) 

D. What Is Done with Student Questions 

1. Are questions answered in a direa and understandable manner? 

2. Are questions answered politely and enthusiastically? 

E. What Is Done with Student Responses 

1. How long does the instructor pause for student responses (for- 
mulating answers to difficult questions takes a few minutes)? 

2. Does he or she use verbal reinforcement? 

3. Does he or she use nonverbal responses (e.g., smile, nod, puzzled 
look)? 

4. Does he or she repeat-answers when necessary so the entire class 
can hear? 

5. Is he or she receptive to student suggestions or viewpoints con- 
trary' to his or her ov^ti? 

Teacher's Presentation Styles 
A Verbal Communications 

1. Can the instructor's voice be clearly heard? 

2. Does he or she raise or lower voice for variety and emphasis? 

3. Is the rate of speech appropriate? Tck) fast or too slow? Appropriate 
for note taking? 

4. Are speech fillers (e.g., "you know" or "in fact") distracting? 

5. Does the instructor talk to the class, not to chalkboard or ceiling? 

B. Noni 'erhai Com mu n ication 

1. Does the instructor look directly at students? 

2. Does he or she scan the class when asking or responding to 
questions? 

3. Does he or she fcxus on particular students or sides of the rcx)m? 

4. Do facial and bcxiy movements contradict speech or expressed 
intentions? 

5. Does the instmctor use facial expressions (such as raised eye 
brows), bcxiy posture (sitting, standing, folding arms), or body 
motions (proximity lo students, clenched fisLs, pointing) to sustain 
student interest? 

Students' Behaviors 

1 . Wliat arc the note taking patterns in the class (do students take 
few notes, write dcwi everything. v^Titc dov^n what in.stmctor puts 
on the board, lean over to copy otlicrs' notes in order to keep up)? 



2. Are students listening attentively, leaning forward, slumped back 
in desks, heads on hands? 

3. Do students listen or talk when other students or the instructor 
are involved in discussion? 

4. How aaively are students involved (asking questions, doing home- 
work. dcKxiling on notebooks, looking out the window)? 

5. Are there behaviors that are out of the mainstream of class activity 
(random conversations among students, reading materials not rele- 
vant to class, passing notes)? 

i\'o(e. Adapted from "An Aj^proach to Colleague E\'aluation of Classrcxim 
Instruction" by M.D. S^)rcinelli. \9S4. Journal of Instructional Det'elopmenf 
''( i ). pp. 14 16, 1'sed by permission. 

nGURE4 
Post-Observation Conference Guide 

1. In geneml, how do you think the class went? 

2. What do you think about your teaching during the class? 

3. Did students accomplish the goals you had planned for the class? 

4. Is there anything that worked well for you in class today— that 
you particularly liked? Does it usually go well? 

5. Is there amthing that did not work well— that you disliked about 
the way the class went? Is this typically a problem area for you? 

6. What were your leaching strengths? Did you notice an>thing you 
improved or any personal goals you met? 

VC^ai were your teaching problems— areas that still need 
improvement? 

8. Do you have any suggestions ^r strategies for improvement? 

Mote Adapted from "An Approach to Colleague Evaluation of Classroom 
Instruction" by M.O. S^)rcinelli. 19^1. Journal of Instn4aionai Det'elopment 
"^(4 ), p. 16. Reprinted by permission. 

Selection of observers 

V'tiether collaborators—teacher and one or more observers— 
come from the same, related, or different disciplines probably 
depends on several factors, including: the puri")Ose for which 
the assessment is conducted, participants' expectations from 
the observation process, specific aspeds of teaching faculty 
seek to improve, and each teacher's comfort level with dif 
fercnt collaborative arrangements It may be that all of these 
arrangements have something to offer when the purpose of 
evaluation is instaictional improvement, but there is far from 
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full agreement on this issue. 

Scriven (1980), Mathias and Rutherford (1982a, 1982b), 
Shulman (1993). and Sorcinelli (1984) adamantly argue that 
colleagues familiar with course content should assess a peer's 
teaching, because it is ia course content that faculty have 
more expertise than students, academic administrators, and 
teaching consultants. That view is expressed by Sorcinelli: 

Put simply, a colleague from one's own or a related depart- 
ment is in the most advantageous position to ohsen>e and 
ei^aluate aspects of the instructor's mastery and selection 
of course content as well as the currency or importance of 
that content within the discipline. Judgments about issues 
such as exhibited knowledge of the content, and preset- 
tation of the origin of ideas and concepts, current devel- 
opments in the field, and the appropriate depth and breadth 
of material cannot be judged adequately by obsen>ers ivith 
limited or no content expertise. It is these tough but impor- 
tant criteria that classroom visitation progratns need to 
address {mAA;^. 12). 

Other scholars either believe there are ad\'antages in having 
observers from nonrelated fields of stud>' (Heller 1989; 
Menges 1987; Shatzky and Silberman 1986) or suggest there 
is a place for obser\'ers from the same and different disci- 
plines (Braskamp 1978; Weimer 1990; Weimer, Kerns, and 
Parrett 1988). The first of these views is argued by Menges: 

When feedback deals with [how content is presented] , a 
colleague's detailed knowledge of course content may 
hinder rather than help. Conversations tend to focus on sub- 
stantive details which are less pertinent than data about 
teacher or student behavior One task of colleague observers 
is to take the role of naive learner, hut it is ei^m more dif- 
ficult for a colleague from the same discipline to assume 
that role than it is for one from a distant discipline ( 1987, 
p. 86). 

In that connection. Weimer. Kerns, and Parrett add: 

There are some argume?its in favour of colleague observers 
from outside the discipline Tha^e familiar with the content 
oi^r-emphasise it in relation to the rest of the instructional 
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eroit. ff Uk' contcfit is inifc4f/iiluo\ the ohsenvr tends toicard 
the opposite extreme coui co/isec/netitly focifses more easily 
on prescfitatiofUil strategies citid tecbniques. To say colleague 
ohsenvrs are totally inuiiuilified because they do not knoiv 
the discipline denies the ralidOy' of their Irmg experience as 
students and instructors They know what it is like to take 
courses outside o/ie's acadenu'c area. This mecnis they can 
reminisce and project. "If I were a sti/dent in this course, 
I think that so nuuiy required readings would dampen ?ny 
enthusiasm'' { 1988. p. 288). 

Woimcr. in c|ualil\ in5» thai \ icw . at least to a degree, contends: 

hi so)ne situations, /hai ing observers J ro))i the same dis- 
cipline/ does help W hen knowledge of the content makes 
a contrihiition to improrenwnt efforts, it is knowledge of 
the material from a general rather than specif ic perspect ire. 

The way contetit is ''shaped and ordered" by the rarious 
disciplitws does hare instrifctit^nal implications, and col- 
league understandifig of ti.fose content configurations can 
co}itribute to certain kinds of in.^tructional decisitms ( 1990. 
p. 118). 

There may be pra;iieal reasons as well for lia\'in4» outside 
oKservcrs. asWeiiner notes: 

/^airing faculty across departments reduces anxiet}' and 
helps to ensiire that the focus is on teaching processes as 
opposed to content. Confidentiality^ is also ecwer to protect 
if the colleague is fnmi across campus, not just down the 
hall Moreoi er not ktioiring the contetit encourages the col- 
league t(f lieu the instruction from that l ety important stu 
dent perspectire. ''How would i he responding if I were 
rei{uired to take this class}' When was / clecny confused 
about the contefit^' When did I find }ny attention leaning}''' 
{ 1990. p 1 19) 

Tsing olrseiAvrs from dillerent academic disciplines may also 
eliminate, or at least rednce. potential coiitlicts of interest aris 
ing tix)m havini; the same faculty involved in both siimmative 
and formative evaluation. It might also be possible to have 
lower level courses observed by colleagues From different 
fields of study and upper level (ourses b\ faculty from the 
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same discipline. 

When evaluation is conducted for the purpose of instruc- 
tional improvement, Weimer, Kerns, and Parrett recommend: 

// the colleague obsenfer is to acquire data related to course 
content, its propriety, currency, level of complexity, etc, then 
the observer must be familiar with the content. If the interest 
is more presentational, instructor enthusiasm, organisation, 
impartiality, etc., then the peer observer must be trained in 
observational techniques but familiarity with the content 
is not required {1988, p. 288). 

We take essentially the same view, finding it consistent, for 
the most part, with Shulman s "knowledge and teaching" con- 
struct (1987). We recommend, then, that reviewers come from 
different disciplines when faculty seek to improve presen- 
tational skills, but from the same or a closely related discipline 
when facult)^ want to strengthen aspects of teaching related 
to course content. Despite a certain surface attractiveness for, 
and perhaps expediency of, routinely selecting peer observers 
from nonrelated fields of stud>\ we believe that the vast major- 
ity of critical teaching incidents are interdependently content- 
and context-bound, requiring analysis from and the assistance 
of colleagues with considerable expertise in the field of study. 
To work successfijlK'. we caution, such an arrangement man- 
dates "good faith" efforts from participants. We are confident 
that faculty can, and will, work in a spirit of magnanimity to 
improve each other's teaching and to elevate the role of 
instaiaion to its rightful lofty position in the academy. 

Procedures 

'I\\'o other, related procedural issues should be considered. 
The first is how often colleagues should visit another faculty 
.members class. The second is how long a collaborative rela- 
tionship should continue. 

In order to be reliable and valid, the number of classroom 
obserN^ations must be sufficient to assure that an instructor's 
typical teaching has been sampled (Braskamp 1978; Centra 
1975). In practice, the number of visits has ranged from two 
per semester (Bell Dobson, and Gram 1977; Sweeney and 
Gaisha 1979) tv) weekly (Katz and Henry 1988) to ever)' class 
meeting (Hlbow 1980; Rorschach and Wliitney 1986; Shatzky 
and Silberman 1986). 
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In practice, nearly all successful collaborative relationships 
have continued over a period of at least a semester. Katz and 
Henry, however, suggest that: 

// is desirable for the tivo colleagues to work together for at 
least two semesters because the effect is cumulative, and 
frequently it snowballs. In our experience a third semester 
of work has proved especially beneficial because, with the 
intenfal of a summer, thoughts and attitudes consolidate 
In our experience, when we ivalked into the class of a col- 
league with whom we had worked for two previous semes- 
ters, we were struck by the feeling of good ivill and enthu- 
siasm that the new group of students exuded, a consequence 
of the different approach that the professor had developed 
during the past year's /^w^(1988, pp. 1516). 

Limitations and criticism 

Despite general support for formative peer observation, 
detractors contend that there are limits to what can be 
obser\'ed and that there are several potential threats to its reli- 
ability' and validity. Wood ( 1977, 1978) describes ways in 
which the process can be biased; 

1. /Vss(KiaticMi (faculty who have close professional and/or 
personal associations aire more likely to rate each other 
higher than those with whom they are asscK'iated less 
frecjuently); 

2. Visibility ( faculty whose offices are k^ated near the central 
office are more likely to be rated higher than those whose 
offices are kx.*aied in more remote areas); 

3. Lack of independence between ratings for teaching and 
research (faculty who are rated high on research also are 
likely to be rated high on teaching); 

■i. Lack of independence betvswn ratings for teaching and 
service (teachers who are rated high on ser\nce also are 
likely to be rated high on teaching); 

"S. Uck of independence between ratings for teaching and 
the number of credit hours taught (faculty who teach 
heavier class loads are more likely to be rated higher than 
those who teach lighter k^ads); 

6. Uick of independencx^ between ratings for teaching and 
nimibcr of graduate courses taught (faculty who teach 
larger numbers of graduate courses are more likely to be 
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rated higher than those who teach fewer graduate 
courses); 

7. Faculty who teach electh e ccuirses are more likely to be 
rated higher than th(\se who teach required courses; and 

8. Lack of independence between academic rank and ratings 
for teaching (facult\' who are at the higher professorial 
ranks are likely to be rated higher on teaching than those 
at lower ranks). 

Centra has identified other potential threats to the reliabilit\' 
and validity of peer obser\'atjon, noting that it has low inter- 
rater reliability, that faculty- are more generous in their ratings 
than students, and that attaining a large-enough sample of 
classroom behaviors in order io make accurate generalizations 
may be prohibitively time-consuming ( 1975). >X''hile conclud- 
ing that these problems are difficult though not insurmount- 
able. Centra cautions that these factors should be carefully 
considered before peer obser\*ation is put into place. 

Most of the concerns and criticisms of Bergman ( 1979, 
1980), Centra ( 1975), and VCx)od (197*^, 1978) regarding the 
reliability and validity' of peer obser\*ation are made with 
respect to summative evaluation. Nevertheless, some of their 
concerns also are applicable to formative evaluation. VC'liile 
some have said such concerns are less important in formati\'e 
evaluation than in summative, we would argue that regardless 
of the purpose, e\*aluators must strive to pro\*ide information 
that is accurate, fair (free of prejudice and or ulterior motive), 
and, if possible, helpful to faculty- in improving their teaching. 
But because faculty' are not in complete agreement about 
''effective teaching," assessment ma>' appear inaccurate, 
biased, or too subjecti\'e to faculty'. 

Because of honest differences of opinion on some tough 
epistemological issues, not because evaluators are incompe- 
tent, biased, or mean-spirited, reliability ccx^fTicients may be 
low^er than hoped. As a reviewer of this report succinctly put 
it, '*The bulw^ark of successful (formative) peer evaluation is 
an underlying commitment to building a culture of teaching, 
a collcgial, mutually supportive, exploratory* community inter 
ested in the teaching learning dynamic," one prizing candid, 
free, and hc^nest discussion and debate of epistemological 
issues affecting teaching, learning, and assessment. 

Direct cla,ssroom obser\-ation has been lashioncd in a \a 
riety of ways at colleges and universities where it has been 
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employed, but it is not the only method of peer evaluation 
that has been used. These other methods include class video- 
taping, course-material evaluating, and assessing instaictor 
evaluations of the academic work of students. 

Videotaping of Classes 

Compared with the volume of literature on classroom obser- 
\'ation. relatively little attention has been paid to how video- 
taping can contribute to instructional improvement. Never- 
theless, its advocates tell us what can be accomplished by 
\'ideo playback/ feedback, how to implement videotaping pro- 
grams, and how to deal with its potential dangers. 

Obfectives 

To many students of faculty ex aluation, videotaping of classes 
is seen as an alternati ■ c to classroom obser\'ation. In some 
respects, it is, since videotaping also can provide useful infor 
mation about what is occurring in a classroom and suggest 
wa\'S in which instruction might be improved. 

Justifying the use of video playback, feedback in formative 
evaluation is more compelling, however, when it can be 
shown that it should be employed in addition to classroom 
c:)bservation. Tliat argument is made persuasi\'el>' by Perlberg: 

The Kniquc equalities of video playback, and in particular 
its authenticity afid high reliability, make it a powerful 
mediator in its own ri^ht atid an important "helper'' to all 
other feedback sources /When a teacher has difficulty 
accepting feedback from students, peers, administrators, 
or teaching consultants!, video recordings, giving both audio 
cind video feedback, could validate feedback from [these] 
other sources. \X1h i all sources of feedback correlate, the 
person is faced with "reality in its nakedness. " which dif- 
ficult to deny^mh. pp 6S6 V). 

Lichty and Peterson also note that \'ideotaping can provide 
information that is difficult to obtain using other methods: 

.sy//c(' the video tapes are a permanent record of the faculty* 
member's teaching performance, this technique of peer eva 
luation provides several added dimensions to the measuring 
of teaching effertiivness. Hrst, the strong points of each 
teachers technique can he visualized and dLsseminated both 
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to the department and interested outside observers. No 
longer will good teaching techniques die after delivery. 
Second, the weak points of each teacher's techniques may 
be s}>stematically retnewed and studied for future correc- 
tion. Third, a teacher may compare past video tapes for 
signs of improvement or decay in his or her classroom 
manner (1979, p. 5). 

A number of scholars have commented more specifically on 
what informed peers could look for as they view tapes of 
classroom teaching. Dressel observes that it might be used 
to illustrate "weaknesses in delivery, in expression, in empha- 
sis, and in attention to students— all of which can be 
improved" (1976, p. 351). Smith and others add that in addi- 
tion to weaknesses in presentation, videotaping can provide 
useful clues to student responses to what was presented 
(1988). 

Craig, Redfield, and Galluzzo envision an even greater role 
for videotaping (1986). They recommend that it be used in 
"stimulated recall interviews," self-reports in which a video- 
tape of a class is played and stopped periodically for students 
to report what they thought about and how they reacted to 
specific incidents at strategic points during the class session. 
The same thinkers suggest that peers are well-qualified to 
assist a colleague in interpreting the information provided 
by the stimulated recall interview. It should be emphasized, 
though, that "interviews with students should be construed 
as the teacher's inquiry into how learning comes about, rather 
than assessments of the goodness or badness of the course 
or the teacher. Interview questions should incorporate sug- 
gestions from the teacher, and the teacher should conduct 
at least some of the interviews" (Menges 1991, p. 34). 

Procedures 

Five practical issues which almost certainly will affect the sue 
cess of videotaping of classes in relation to teaching improve- 
ment need to be considered. Tliese issues are: how to get 
teachers to participate, how long a playback/feedback session 
should last, how much time should elapse between taping 
and playback/feedback, what type of participant training 
should be provided, and how many tapings of a class will be 
required to determine typical teaching. 
Studies by Britt (1982) and Keig ( 1991 ) have shown that 
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faculty are iiuich less inclined to lake pan in videotaping than 
in other methods of cvaiuaiion. Indeed, as Pcrlberg notes, 
for teaching consultants, '*the major problem is how to mot 
ivate faculty to be involved in intense experiential teacher 
training programs, including (videotaping), which are per- 
ceived as stressftjl experiences" Perlberg recommends that 
the initial appeal for participation should be made to partic 
ularly conscientious teachers who want to improve their teach- 
ing and perhaps also to those with a "natural curiosity to see 
themselves as others see thenV ( 1983. p. 657 ). 

lx)nger playback feedback sessions seem to be more effec- 
tix e than shorter ones. Most effective of all are sessions lasting 
40 minutes or longer. Sessions lasting 30 minutes or more 
are more effective in affecting change than those lasting 20 
minutes or less (Perlberg 1983). 

Perlberg recommends that a professor look at the videotape 
of his or her performance on two different occasions (1983). 
A partial viewing, according to Perlberg, should cx'cur imme- 
diately following the taping and should be "aimed at reducing 
stress :uid anxiety through reinforcement of the positive con- 
tent, and fcK'using on points for fuither contemplation" 
(p. 648). The second, more extensive and intensive, playback,' 
feedback session "will thus be free of many of the stresses 
following the recording. Having a preliminar\' [playback] also 
provides the consultant and the [professor] with time for con 
temptation and perspective" (p. 648). 

Perlberg insists that participants must he trained in how 
to transmit findings to colleagues. Fie states that "one cannot 
emphasi/.e too strongly the importance of receiving the neces- 
sar\- consultancy skills for effective use of [video playback 
feedback] in higher education" (p. 658). 

The number of classes that should be videotaped is a ques 
tion raised in the literature hut not actually answered (1983). 
Too few tapings would surely limit the representativeness of 
the leaching sample and its accuracy. ^Hiis issue is addressed 
more fully in the section on disincentives. 

Model 

McOaniers three stage model of formative evaluation of teach 
ing includes a prominent role for \ ideo playback feedback, 
its three stages in\()Ke ha\'ing facults' establish standards ot 
effective teaching; evaluate \'ideotapes of their teaching, with 
the heli^ of a teaching consultant, against the teaching stan 
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dards; and view tapes of successful teaching of other partic- 
ipants and "discuss how the episodes illustrate the teaching 
behaviors they previously identified" (1987, p. 99). 

Potential iiangers and caveats 

The self-confrontational nature of viewing tapes of and recei\' 
ing feedback from one's teaching either can be helpful in 
improving performance or a debilitating experience. The 
manner in which a videotape is presented to a teacher may 
determine how praise and/or constmctive criticism are 
received. Nearly all writers on videotaping comment on the 
potential dangers of this self-confrontation (Brandenburg, 
Braskamp, and Ory 1979; Braskamp 1978; brcKk 1981; Craig, 
Redfield, and Galluzzo 1986; Dressel, 1976; Fuller and Man- 
ning 1973; Perlberg 1983; Seldin 1984; Smith et al 1988). 
Brandenburg, Braskamp, and Ory express well the sentiments 
of many wTiters when they say that findings from videotapes 
are "especially personal and descriptive; viewing a videotape 
with a colleague is preferable to only the instructor viewing 
it because the colleague can share his/her insights, can pro- 
vide support in this confrontational experience, and suggest 
improvements and changes" (1979> p. 12). 

However, the self-confrontational nature of video pla^'back.'' 
feedback is a salient strength as well as a potential danger. 
Without it, little of value can be accomplished. Perlberg 
explains how much of the potential benefit of videotaping 
can be lost if self confrontation is not judiciously exploited 
by program participants and peer reviewcr(s): 

Knowing that [video playback/feedback J could be ivry' 
stressft^l and at times even hannjul, tfoey choose the easy 
way oiU—tdsing [videotaping/ in a superficial way which 
minimizes arousal and stress. The client t^ies to avoid dL^ 
covery or admission of discrepancies The consultant col- 
It ides in (he defense in order to avoid arousing or panicking 
the client. The video is used mainly as a minvr for observing 
external cosmetic phenomena, or other trivial behavior, 
rather than for focusing on the basic issues at stake. Thus, 
the most powerful available technique for changing behav 
ior w wasted { 1983, p. 6S8). 

Yet, as Brinko admonishes; 

r/'rA'o feedback is not for everyone. In many instances it 
can be a useful tool: in other cases it can he a threatening 




and stress} id experience, actually inbibithig performance 
or even increasing those behaviors which are desired to he 
extinguished. This same reasoning can he applied to all 
methods of feedback: the literature on individual differ- 
ences makes clear thai a wide range of perceptions and 
preferences exist among people in their reactions to feed- 
back and in their learning styles. Thus, different modes of 
feedback will be more informative, meaningful, and rele- 
vant than other modes to different individuals {\99i. p. 
582). 

Wliile intrusive and, to some taculty, tlireatening, videotaping 
can be a powerfu' tool for etlecting changes in teaching 
beliaviors. Trusted colleagues, trained in consultative skills, 
can help each other use video playback feedback, in conjunc 
lion with other methods of evaluation, to improve their 
teaching. 

Direct classroom obsetration and videotaping of classes 
are appropriate for assessing w hat cKCurs when a teacher and 
his or her students interact. We turn now to three methods 
designed to critique prc-interacti\'e and post-interactive teach 
ing events. 

Evaluation of Course Materials 

Students of faculty evaluation generally agree that informed 
peers are ideally suited to assess colleagues course materials. 
Menges, in fact, claims that 'no one is better able than a col 
league to make knowledgeable comments about the accuracy 
and currency of teacher materials' (1987, p. 86). But scholars 
also note that few colleges and universities have integrated 
this method into the e\aluation prcxess ( Cohen and McKea 
chic 1980; Seldin 1984, 1993c; Weimer 1990). Moreover, a 
review of the literature reveals that considerably less attention 
has been paid to evaluation of course materials than to either 
classroom evaluation or videotaping of classes. 

It is not altogether clear why evaluation of course materials 
has been neglect c?d in practice, although some .scholars spec 
ulaie. Seldin. for example, suggests that its limited use may 
be merely one of oversight ( I9H * ). Centra offers three more 
substantive reasons: Course materials are so personal and sub 
jective thai faculty members are noi willing to oju-n the mate 
rials to the same close scrutiny that they gi\ e colleagues in 
review of manuscripts for publication; the time required to 
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review course materials can be better spent on research where 
the extrinsic rewards are usually greater; and it is not worth 
the time because course materials are read "only" by students, 
while published research is there for everyone to read and 
evaluate ( 1986). 

Nevertheless, one academic "is convinced that a great deal 
of unacknowledged brilliance resides in our colleagues' clxss 
room strategies, in their syllabi, in their paper and examina- 
tion topics, and in their paper grading—at all le\'els of instruc- 
tion*! (Miller 1990, p. 53) This assertion suggests that college 
teaching could be improved if peer review of course materials 
and assessment of instnictor evaluation of students' academic 
work were a more common practice. At this point in the 
report, we look at what might he accomplished by peer 
reviev.' of cor';e materials, what materials could be jxamined. 
how this method of evaluation might be inplemented, and 
what the limitations of this method are. 

Objectives 

McCarthey and Peterson not only explain, in broad terms, 
what might be accomplished by peer review of course mate- 
rials but also suggest, somewhat more subtly, why assessment 
of course materials should be combined with other methods 
of peer evaluation if the full range of teachers' competencies 
is to be ascertained. 71ie\' write; 

Teacher materials yield factual and objective data for peers 
to judge. These materials provide an oveniew of the cur 
riculum taught, information about teaching strategies, and 
details about assig)imerus given. Materials can indicate types 
of communication with students . . . and peers, the kvid 
of management system used, and resources provided to stit 
dents. Peer review of materials gives teachers the opportunit}' 
to demonstrate excellence through the content and activities 
of the classroom, and to reflect teacher individuality. Finally, 
there is a plausible logical connection between quality mate 
rials and qualit}' classroom performance for many, but not 
all, teachers ( 1988, p. 26 1 ). 

Process 

Several scholars haw identified competencies faculty should 
ha\c in course planning, instnictional design, and test prep 
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aration that could be assessed by peer review of course mate- 
rials (Aleamoni 1981, 1984; Brandenburg, Braskamp, and Ory 
1979; Braskamp 1978; Cancelli 1987; Centra 1986; Cohen and 
McKeachie 1980; Dienst 1981; Ecken 1950; McCarthey and 
Peterson 1988; Scriven 1980, 1983, 1985; Seldin 1984; Smith 
1985; Smith et al. 1988; Weimer 1990). Figure 5 is a compi- 
lation of the work of these scholars. In the figure, the cate- 
gories are organized by the medium from which information 
could be obtained. In some instances, these competencies 
could be placed under more than one heading. In other 
instances, the competencies may be appropriate for some aca- 
demic disciplines but not for others. 

FIGURES 

Guide to Evaluation of Course Materials 

Syllahtis 

Instructor demonstrates command of course content 
The breadth and depth of course content are appropriate 
Emphasis and time given to each major topic is appropriate 
The course content is an adequate prerequisite to other courses 
Course objectives are specific enough to constitute a really useful 

guide in selecting and organizing class activities 
Content is organized logically, in a way that seems meaningful to 

students at this level of preparation 
The sequence of topics to be covered is appropriate 
Difficulty- level of the course is appropriate for its curricular level 

and for the students enrolled 
Goals and objectives are stated clearly 

Course goals and objectives are in line with department and/ or col 

lege goals and objectives 
Syllabus helps orient students to their learning tasks 
MethcxKs) of instruction is/are suitable for course goals and 

objectives 

Student work requirements for the course are appropriate 
Standards used for grading are communicated clearly to students 
Syllabus is revised pericxiically to reflect recent scholarship, changing 

student needs, more sophisticated thinking about the teaching 

of the course, etc. 
Content duplicates/does not duplicate that of other course(s) 

Rc(uU}igs (Old Other I.vuniiti^ At tit Itics 
llie work of recognized authorities in the field is included in 
readings 
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Basic concepts of the content area are covered in readings and/or 

other learning activities 
Readings reflect discriminating choice of books and/or journal 

articles 

Content is up-to-date; instructional materials include recent devel- 
opments in content 

Headings are appropriate for the level of the course 

Reading assignm.ents require an appropriate amount of time and 
effort to complete 

Course materials challenge and stimulate students intellectually 

Handouts and other learning aids are suitable adjuncts to primar\' 
instructional materials 

Media materials (e.g , films, videotapes, audiotapes, multimedia, com^ 
puter programs) are used in appropriate ways 

Community resources are used appropriately to supplement class 
presentations and other learning aaivities 

Tests, Papers, Projects, Presentations, and Other Assigned Academic 
Work 

Test content is representative of the content of the unit under study 

(lest exhibits content validity) 
Test items are clear and weli-written 

Tests require appropriate lower-level and higher-level cognitive skills 
Criteria for the grading of tests (and other assignments) are approp- 
riate and clearly communicated to students 
Tests are reasonable in length and difficulty- 
Assigned academic work is appropriate to course level 
Students apply principles learned from class presentations and read 
ings in papers, projects, presentations, and other assigned aca 
demic work and on tests 
Assigned at.:ademic work can be tailored to meet itidividual student 

needs and interests 
Assigned academic work requires reasonable time and effort to 
complete 

In virtually all of his writings on faculty evaluation, Scriven 
emphasizes the need for teachers to be assessed on their 
knowledge of course content. He says, for example, that: 

There must be careful examination of the quality and pro- 
fessionality of content and process: the three qualities here 
are currency, correctness, and comprehensiveness. Ratings 
must he made on the basis of a sample of 1) the materials 
protnded, 2) the texts required and recommended, 3 ) the 
exams, 4) the term paper topics, 5) the student perfonuan 
ces on the preceding, 6) the instructor's performance in 
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grading student ivork, 7) the instructor's performance in 
justifying the grade and providing other helpful feedback 
\Scriven 1980. p, 13). 

not a strong advocate of peer evaluation in general, 
Scriven concedes that evaluation of course materials "is the 
one place where peer evaluation of a limited kind is appro- 
priate^^ (1980, p. 13). At least some of the content knowledge 
listed above is surely among that informed peers could assess 



Procedures 

Smith and others recommend that the procedure for assessing 
course materials be similar to that described for direct class- 
room obser\'ation (1988). They suggest having a panel of col- 
leagues: independently evaluate a range of the teacher's 
course materials (syllabus, textbook(s), reading list, tests, 
etc.), based on predetermined criteria (e.g., currency, rele- 
vancy-, accreditation standards); meet to discuss their findings 
and, Vf necessary, to arrive at some degree of consensus; meet 
with the course instructor to discuss the findings, clarify infor- 
mation, and provide feedback; iind write a summary report 
of findings and recommendations. After the process is com- 
pleted, the course instaictor is asked to consider the findings 
and then either plan for and implement changes or explain 
his or her decision not to do so. 



Limitations 

There are limitations, however, to what can be learned from 
evaluation of course materials, particularly when such review 
is conducted independently of other methods of evaluation. 
McCarthey and Peterson note that: 

Fi vn when peer ei^aluation does not involve classroom visits, 
but AS restricted to a review of materials, there are signif- 
icant resen^ations. Not all teachers are effective through their 
materials: some exceUent teachers work with spartan pro 
visiom. The time cost of materials assembly may not be jus- 
tified by the increase in information provided. Some 
teachers seriously object to peer review, while the connection 
between peer revieiv and teacher quality is not always 
direct l\'er revieiv is intertnediate in expense relative to 
other data sources: student sunxys, for example, cost less 
money and time. Mere collections of good materials have 
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little direct relation to quality implementation, interactions 
with students, and creativity in presentation (19SS, p. 261). 

Faculty colleagues are in a better position than students, 
administrators, and teaching consultants to evaluate course 
materials. Informed peers can help colleagues look at these 
materials in context of objectives, presentation, and outcomes. 
Peer evaluation of such materials, used in conjunction with 
other methods of evaluation, including the one we look at 
next, has the potential to improve reaching and student 
learning. 

Assessment of Instructor Evaluations of the Academic 
Work of Students 

Assessment of instructor evaluations of the academic work 
of students is, in one sense, a dimension of e\^luation of 
course materials. In another sense, it is considerably different 
because its focus is on a kind of teacher performance occur- 
ring, in large pait, following deliver}^ of instruction rather than 
in course planning or at times wiien students and a teacher 
are interacting. Since the two methods of evaluation may have 
different impacts on students, evaluation of instructor-graded 
student assignments is treated in its own right here. 

like evaluation of course materials, assessment of instaictor 
evaluations of students' academic work has only occasionally 
been employed in higher education (Seldin 1984, 1993c). 
And even where these methods are addressed in the literature, 
disentangli.ng information about them often is difficult. That 
problem notwithstanding, we look at what might be accomp- 
lished by assessment of instructor evaluations of the academic 
work of students, at what might be examined, at how it could 
be implemented, and at some of its limitations. 

Objectives 

Tlioughtful comments concerning an instnictor's responsi- 
bilities in facilitating student learning, in a sense a rationale 
for including peer review of instnictor-graded student assign- 
ments as part of the process of evaluation, have been aitic- 
ulated by Dressel. 

/It Ls an instructors responsibility^] to provide the student 
with satisfaction through a sense of progress. The respon- 
sibility' requires pointing out to the student both successes 



and deficiencies. . . . Evaluation for feedback aftd moti- 
vation is an essential component of good teaching. [Little 
can! replace the personal commendation of an admired 
teacher. Praise or the regard of others is a potent motivator 

[Yet] an indispensable aspect of learning is the recognition 
and admission of error, combined with the ability to profit 
from error Faili^re must come to be regarded as a chal- 
lenge, not as a disabling or uncorrectable event which 
impedes further progress {1976, p. 343). 

Dressel implies that a teacher s feedback to students requires 
a balance between praise and constaictive criticism, a point 
not easily achieved. The academic work of students includes 
an array of activities depending on their fields of study: tests, 
papers, book reviews, projects, presentations, performances, 
laborator>' and studio work, field work, and homework. All 
of these present opportunities for teachers to give relevant 
information to students about their academic performance, 
opportunities that are not always used to full advantage. 
McKeachie, for example, observes that teachers too rarely 
exploit the potential of tests in providing meaningful feedback 
to students and suggests that if facult>' colleagues were to 
review graded tests (and other assignments), they could assist 
each other in providing accurate and appropriate feedback 
to students (1986). 

Bryant also provides grist for peer review of instrucnor eval- 
uations of students* academic work: 

The real proof of a teacher's competence is how much his 
students learn and what the teacher expects of them. Accord- 
ingly, the examinatiom given fej- a professor should be scru- 
tinized . . . and his students' papers should be read [by col- 
leagues} carefully if a professor obtains a high level of 
performance from his students, he may be an effective 
teacher, whether his students consider him a good buddy 
or not. Admittedly, . . . et^aiuations would have to be made 
with intelligence, but one hopes there is still some of that 
quaiit}' available in our universities {1961, p. 329). 

Feedback need not be confined to written comments. A con- 
ference in which an instructor candidly and tactfully discusses 
the strengths and weaknesses of an assignment with a student 
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is another. Still another means is through audiotape, as Kalz 
and Henr\- describe: 



One prof f'ssor . . . began his course with the practice, con 
ti>ineu throughout the semester, of dictating bis responses 
to the written work of hL^ students on a tape, (Each sUident 
provided a tape for repeated dictations ). This practice not 
only aiioived for a more personal and relaxed commun- 
ication—stnde?its commented favorably on the tape, which 
always started out ''Dear Jim'' or "Dear Jane'^— but also 
eschewed the usually greater finality* of written comments 
(1988, p. 16). * 

Facult\' need to de\*elop skills in giving feedback to studcr.ts 
and to seixc opportunities for offering it in appropriate ways. 

Process 

A number of scholars attempt to explain with some specincity 
what peer re\ iewers should look for when they read instruc- 
tors' evaluations of the academic work of students (Braskamp 
I9':'8; Dienst l981;Scriven 1980; Seldin 1984 ). A compilation 
of their suggestions as well as some of our own ideas are pro- 
vicied as Figure 6. (We are indebted to Scri\'en i 19801 for 
items appearing under the tlrst entry.) 

In insisting that more attention be given to the way in 
which faculty prepare and grade tests, Scri\'en states: 

There are profeswnally required standards here, with which 
virtually no faculty member at universities hare the slightest 
familiarity*. a remedy for this, administrators should 
request that as a normal part of the process oj talking about 
self improvement, the instructor fills out a form indicating 
how papers are in fact graded( 1980. p. 15). 

Facult\- whose knowledge on these matters is deficient or 
limited should be referred to colleagues with expertise in 
tests and measurements. 

Procedures 

A procedure similar to the one proposed by Smith and others 
for peer evaluation of course materials could also be em 
ployed for a.s.sessment of iastructor ex'aluations of students' 
academic work ( 1^)88). The piocedure might inxolve having 




FIGURE 6 

Guide to Assessment of Instructor Evaluations of 
the Academic Work of Students 

Tests 

Tesis arc graded in a fair arid a Kisisicni maucr: 

• "blind" (pnncciing ihc anommii>' of suidenis w hen paper.s arc 
graded ) 

• qucsiiDii by tiiicsiiDii. raihcr ihan icsi h\ icsi do a\oid ihc "lialo 
cffccl" rcsiiliing from ha\ ing read a pariicularK- good, or had. first 
answer of a suidcm jusi bef.)re reading another answer) 

• the first few graded answers are read again after reading all answers 
( to see if grading standards have gone up or down ) 

• papers are shuffled after reading answers in eaeh question ( so that 
students fare equally in the teaehcr's initial optimism or fatigue 
as ansxvers are read ) 

Responses indicate that students use higher order, a.s well a.s lower 

level, thinking 
Teacher provides constructi\ e feedback to students 
Teacher uses a \ariet\- of means (oral as well as written ) lo pro\ide 

feedback U) students 
Standards used for grading are communicated to and understood 

by students 

'lests are graded and returned promptK- to students 

Papers. Projccfs. Prcscniaiions, citid Other Acadtntik Whrh 
All assignments are evaluated in a fair and consistent manner 
Academic work submitted by students for e\aluaiion indicates that 

students employ higber ie\ el. as well as lower le\el. thinking 
'leacher provides constructiw feedback to students on all iicademic 

work submitted for e\-aluation 
Teai'her em)M( a s a \ ariet\- of means written and oral in providing 

feedback to students 
Standards used for e\alunting different f(ums of academic work are 

clearl\- comnuinicated to and understood by students 
Assignments are graded and returned promptK- to students 
Assignments submitted h\ students are ( )f acceptable ( ( )r better ) 
ijuality 

a panel of eolleagues: independently evaluate a representaii\ e 
sample of students* eoiirse work, assessing the grading of dic 
work and die cjnalitx' of tbe feedback sUidenls received: meel 
to discuss ib.e finLiings and U) make lentaiive recommenda 
tions; meel with the course instriieior to share and elarlK* 
information and lopivseni findings and iveommendations: 



( .(dldharatu e Peer Rvncw 

81 



and write a summary report of the review. Following the 
meeting with the panel, the course instmaor would be given 
an opportunity to respond to the report, agreeing to imple- 
ment changes or explaining a decision not to do so. 

Limitations 

When colleagues are called upon to assess instructor evalua- 
tions of the academic work of students, they may be tempted 
to evaluate student learning rather than to critique the appro- 
priateness of the grading of the student work and the instruc- 
tor's feedback to students. Therein lies a potential problem. 

Scholars warn that assessing student achievement on the 
basis of instruaor-graded student assignments (or even stand- 
ardized tests), be it by colleagues or administrators, involves 
difficult psychometric problems. In this regard. Centra con- 
cludes that "there is at this time no evidence that these assess- 
ments will be valid or reliable" (1986, p. 4). The same author 
notes that this kind of evaluation is especially problematical, 
because variables besides teaching affea student learning. 
In expanding on and clarifying that argument, Menges 
explains: 

Some argue (hat changes in students constitute the infor- 
mation of greatest relei^ance for teaching improvements. 
Examinations, papers, lab reports, and other- graded work 
are undoubtedly infonnatii^e, as is information about stu- 
dents' study habits and their scores on staruiardized tests. 
The major problems with using information about learning 
to improve teaching are that graded ux)rk is an incomplete 
representation of intended learning outcomes, it is difficult 
to connect particular features of teaching with specific 
learning outcomes, and some important influences on 
learning are beyond the teacher's control (1991, p. 30). 

Cohen and McKeachie take a slightly different, though still 
cautious, view (1980). The>' conclude that "colleagues, who 
have a sense of typical student performance, are in the best 
position to judge the instructional impact on students. As of 
now, though, such judgments are qualitative in nature and 
can be best used for supplementing other data" (p. 151). 

The temptation to place loo much emphasis on student 
outcomes can be avoided in large part by limiting assessment 
of instmctor evaluations of the academic work of students 
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to how well teachers grade their students' work and to the 
quality of feedback students receive. Another way to reduce 
the likelihood that one aspect of the procedure will be over- 
emphasized is to employ a variety' of methods and several 
data-providing constituencies in the formative evaluation 
process. 

The potential of assessment of instaictor evaluations of the 
academic work of students is, in many respects, an unexplored 
area in the formative evaluation of teaching. It is possible that 
it may have a more significant place in the future, especially 
if faculty are encouraged to develop teaching portfolios as 
part of a process to impro\'e teaching. 

Teaching Portfolios 

Teaching portfolios are collections of materials assembled 
by faculty members to dcx:ument what and how they teach 
and to explain why th^y teach as they do. Seldin suggests that 
a teaching portfolio is a means by which faculty "display their 
teaching accomplishments for examination by others. And, 
in the process, [portfolios] contribute both to solid personnel 
decisions and to the professional development of individual 
faculty' members" (1991, p. 3). 

Wlicn assembled for the puipcxse of formative evaluation, 
Fxigerton, Hutchings, and Quinlan obsen-c: 

Portjolios ifwitc faculty participation in the examination 
oJouQ another s teaching [emphasis theirs]. Faculty can 
work collaboratively in constructing their portfolios, they 
can aLso use portfolios windows to view and share per- 
spectives on one another's teaching. Such collaboration is 
almost certain to be powerful when the aim i$\o improve 
teaching /emphasL^ ours] ( 199U p. 3). 

In compiling a portfolio, faculty usually include "artifacts of 
teaching ' such as course syllabi, reading li.sts, tests, and the 
daih' work, papers, and laboratoiy excrci.ses of their students, 
llicy also may inclixic "reproductions and representations 
of vvhai happened** -videotapes, photographs, diaries, jour- 
nals, and the student evaluations of their courses (Edgenon, 
Hulching.s, and Quinlan 1991, p. 7). Tlic 1986 publication. 
The Teaching Dossier, of the Canadian As.s.viation of Univer 
sity Teachers, lists 49 "possible items for inclusion*' under 
the headings "Tlie Products of Good Teaching,** "Material from 
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Oneself," and "Information from Others,*' 

Compiling the materials is only pan of assembling a port- 
folio. An equally important part is the reflection on, or self- 
assessment of, the materials included. As they reflect on the 
materials included, faculty may comment on what worked 
well, what wasn't as successful, what might be tried as alter- 
natives, and what might be modified or discarded. At this 
point in the process, the focus is on why and how, not on 
what. 

The portfolio pro^.'ess has much to commend it— it can be 
comprehensive, can reflea individual uniquenesses, and can 
involve peers in discussion about this central aspect of their 
work. Of course, the use of portfolios is not without problems, 
Shulman captures this tension when he suggests that 
portfolios: 

arc mess)' to construct, cumbersome to store, difficult to 
score, and vulnerable to misrepresentation. But in ways that 
no other assessment method can, portfolios provide a con- 
nection to the contexts and personal histories that charac- 
terize real teaching and make it possible to document the 
unfolding of both teaching and learning over time ( 1988, 
p. 36). 

It is important not to gloss over the barriers that exist to the 
effective use of this technique. Initial acceptance of and com- 
mitment to this process by a faculty is a major potential obsta 
clc. There is no d^ ubt that it can be time-consuming for both 
the person a^ssembling and those who are reviewing it. The 
criteria for evaluation can vary. And perhaps most troubling 
in that our primary concern is formative evaluation, it can be 
difficult to maintain the separation between formative (devel- 
opmental—for teaching improvement) and summative (for 
•personnel decisions) evaluation. Once a portfolio is 
assembled and a^ssessed, the result of the pra:ess is at least 
in the collective mind of those peers involved; it also may 
become a historical artifact in a file, unless stipulated other- 
wise in the guidelines for the prcKess. 

At the same time, it should be noted that multiple examples 
of successpjl implementations exist, Anderson profiles 2S such 
examples in Campus L 'se of the Teaching Portfolio ( 1993), 
One example comes from Gordon College in Wenham, Mass., 
wliere a portfolio- like prcKess has been used in the tenure 
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and promotion decisions for a number of years. This has made 
the transition to a faculty-development portfolio process easier 
than in other places without such a history'. Starting on a small 
scale in 1992 with four faculty, three-quarters of the facult\' 
now voluntarily participate in a portfolio-development 
process. 

The key element in the portfolio, in addition to SN^llabi and 
reflection on their work, is the piece on "lessons learned" 
at Gordon College. The compiled portfolio is reviewed by 
a facult>''development committee and the academic dean, 
who gives one-on-one qualitative feedback. Faculty' partic- 
ipating report satisfaction, noting growth in this aspect of their 
professional lives. Gordon College's experience suggests that 
new efforts start on a small scale, with voluntary participation, 
and, ideally, involve well-respected teachers as mentors in 
the process. 

The objective of peer review of teaching portfolios, peer 
evaluation of course materials, and peer assessment of instaic- 
tor evaluations of the academic work of students is essentially 
the same: better teaching. The differences among them lie 
in how this objective is to be accomplished. 

Our primary intent in discussing the teaching portfolio has 
been to draw parallels between it arid the evaluation of course 
materials and the assessment of instaictor evaluations of the 
academic work of students. We have not attempted to explain 
diUy the potential and limitations of portfolios, since the topic 
has been treated extensively in several recent publications 
(e.g., Anderson 1993; Centra 1993; Edgerton, Hutchings, and 
Quinlan 1991; Hutchings 1993; Seldin 1991. 1993b; and 
others). 

Important as classroom obser\'ation, video playiDack^feed- 
back, evaluation of course materials, assessment of instructor 
evaluations of students' academic work, and analysis of teach 
ing portfolios are in formative peer evaluation of teaching, 
the>' are but five available methcxis. Other methods that could 
be used are those in which the input of faculty and students 
are combined. These include course instaictor and facult)' 
colleague interviews of individual students and/or groups 
of students and stimulated recall inter\'iews in which students 
describe and reflect on specific events that cxvun-ed during 
a class session. 

The methods described in this section present viable oppor 
tunities for improving college teaching. While each of them 



The compiled 
portfolio is 
reviewed by 
a faculty' 
development 
committee and 
the academic 
dearly who 
gives one-on- 
one qualitative 
feedback. 



CoUaborMive Peer Rcrknv 



7/ 



85 



can be used by itself, a combination of these methods— a 
comprehensive approach— is more desirable if the full range 
of teacher competencies is to be critiqued. To have credibility 
with faculty, this comprehensiveness may be indispensable. 

In the following section, we look at a number of program 
examples. While none of them is comprehensive, at least in 
the sense described in this report, we recommend that readers 
study all of them carefulh', for each has worthy elements and 
each has enjoyed success at least on one college or university 
campus. 
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FORMATIVE PEER EVALUATION PROGRAM EXAMPLES 



Tlie methods of formative peer ex'aluation discussed thus tar 
have been put into place at relati\'el\' few colleges and uni- 
versities. Surprisingly, no fully comprehensive programs, as 
we envision them, have been identified. ^X1lile some are 
broader in scope than t)thers, most rely hea\ iK\ or even 
entirely, on one method, usually direct classroom obser\'ation. 
However, the push to have faculty develop teaching portfolios, 
and interest in instructional impro\ement in general, may 
indicate that mc)re inclusive program.s are in the ofling. 

\n this section, we look first at single-institution formati\e 
peer evaluation programs. Since fully comprehensive pro- 
grams have yet to be put in place, we organize our discussion 
of these programs around the m.ethod employed most prom 
inently: direct classroom obser\*atic)n, videotaping of classes, 
and evaluation of course materials and of instructor assess 
ment of students' academic work. Second, we describe two 
multi-institution programs. Third, we assess the strengths and 
weaknesses of the programs described. Fourth, we look at 
their common elements. Finally, we explain how formative 
peer re\iew programs have traditionally been evaluated and 
suggest how this process might be impro\^*d. 

Single-Institution Programs 
Direct classroom observation 

Formative peer observ'ation programs have lieen described 
more often than other methods. Peer review of this t\pe has 
been developed at California State University, Sacramento 
(Stoner and Martin 1993). FA'ergreen State College (Elbow 
1980, 1986), New York University (Rorschach and VCliitney 
1986), Texas Tech University (Skoog 1980), University of Bir- 
mingham, England (Mathias and Rutherford 1982a, 1982b), 
University of Cincinnati (Sweeney 19"76; Sweeney and Grasha 
19"9). University of Kentucky- (Cowen, Davis, and Bird 19*"6). 
Universit\' c^f New York, Cortland (Stat/.ky and Silberman 
1986), University of New York, Stony Brook (K;uz and Heniy 
1988), and Universit\- of South Car)lina (Bell, Dobson, and 
Gram ur~'). 

University of New York, Stony Brook.* At the l'ni\'ersii\' 
of New M)rk, Slon\- Brook (K:it/ and Henr\- 1988 ). two pro 

*\Vc a-li'i to ilus piD^rani as ifu- I nncrsiiy nl Ncw Vnrk. stony Urimk. pro 
j»r.mi. bin WM\/v ihat K;ii/ atul Mcnn"s suidy. a\uM by FllNH aiKl l\»al hmt 
diUon >;rafils. iinuUcxi instnutiona! iniprdxcnicnt ptojcus at 1^ tnsMUttions 
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fessors work together, one in the role of obsen'ed teacher 
and the other as the obsen'er. for at least a semester or, if jX)s- 
siblc. for t\vo or three consecutivt; semesters. Both of the pro 
fessors reguiari\' inten ieu' students about how and what they 
are learning, and the professors meet frecjuently lo discuss 
what is occurring in the class and what they are learning from 
the student inter\-iews. Both professors and the students com- 
plete the Omnibus Personality Inventocy (OPI). the results 
of which are used compare the professors' and students' 
j")referred learning styles. At the end of each semester, both 
professors write rej")orts in which the\' reflect on what they 
have learned about teaching and student learning. Instructor 
e\ aluations of students' academic work can also he used lo 
provide information. 

Wliile (lexible in many respects, the process aLso has pres 
cribed elements. Ideally, as Katz and Heniy point out: 

The coilca^ne rmts the professor's doss once a ireek ( or 
more often if desirable cuid if time permits), meets with the 
professor otice a week coul inten iews three students itidi- 
I id I (ally. The interrieirs are designed to obtain as detailed 
a picti i ) V as possible of u 'hat and how stnden ts learn. These 
i)iterriews provide the professor a) id hui colleague with many 
data on the basis of which to chart what is happening in 
the classroom and what learning the students are doing 
(pp. 10 11). 

The same six students are inter\ iewed each week, three by 
the obser\ ed professor and three by the ()bser\ er. One stu 
dent in each professor's grc;up should ha\'e a learning .style 
(as indicated by the OPI ) similar to that of the ohser\*ed pro 
fessor, a second .student a markedK' different learning .style, 
and the third a learning style .somewhere between the two 
extremes. As Katz and Henn' explain: 

The prime objectiiv of these hiterriews is to gain as detailed 
a picture as is possible of student learning in the course 
under inivstigation. This includes exploration of student 
learning styles, the student's cognitire stage of deivlopment, 
and the student's interests, aspirations, personality, and 
social i ircumstances because all of these can cooperate with 
or defeat learning in the cour,\e (p. 12 ). 




Another approach for eliciting information about course con- 
tent and its presentation is what Craig, Redfield, and Galluzzo 
have called the stimulated recall inter\aew. At Stony Brook, 
the stimulated recall inter\aev.' is used in conjunction with 
classroom observation to: 

iriquire about the students' specific reactions to the class 
session preceding the interview, preferably a class the 
observer has watched. Specific parts of that class may come 
under special scrutiny and students may be asked to talk 
in detail about what went through their mind, what they 
thought and felt as a particularly salient event took place— 
the evenl being a teacher's presentation, other students' con- 
tributions, an interchange, a perception of their own ( 1986, 
p. 13). 

According to Katz and Henry, the program requires four or 
five hours of each professor's time per week. There are, how 
ever, ways in which the program can be streamlined,- includ- 
ing making fewer classroom observations and conducting stu- 
dent interviews less regularly, but, as the authors obser\'e, 
"there is less benefit if less time is spent" (p. 11). 

University of Cincinnati. The program developed for the 
University of Cincinnati and other colleges and universities 
(Sweeney 1976; Sweeney and Grasha 1979) involves three 
collaborating faculty members, each having classes observed 
and each obser\'ing classes of the other team members. The 
program has five phases: goal setting, a first team meeting, 
classrcx)m observation, subsequent team meetings, and pro- 
gram evaluation. 

In the goal-setting phase, each teacher lists his or her 
instaictional objectives for a class to be observed and indi- 
cates how the objectives are related to course goals. At the 
first meeting, team members discuss and clarify the objectives, 
determine the focus of each obser\'ation and the procedures 
by which data will be gathered and reported, and schedule 
the first round of observations. 

When the classroom visitations cKXur. the observers com 
pile relevant information in areas specified by the classroom 
teacher, using the methods and procedures that have i:)een 
agreed upon at the first team meeting. The peer review may 
include methods besides classroom (fecrv'ation if the par 
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licipants agree that other methods could provide useful 
information. 

At meetings where classroom obser\'ations are discussed, 
"obsen^ers are asked to reconstnjct the details of the session 
obser\'ed to establish a common ground of discussion" 
(Sweeney and Grasha 1979, p. 55). They point out positive 
aspects of their colleague s teaching and offer suggestions 
and possible alternatives in areas where they believe perfor- 
mance could be improved. The observed teacher is encour- 
aged to respond to the feedback by asking questions, seeking 
clarification, and commenting on what occurred during the 
class. After the discussion, the observed teacher is asked to 
develop strategies to improve specific aspects c teaching, 
based on what he or she has learned from the obserx^ation 
process and from self assessment. Finally, another classroom 
visit is scheduled. 

After a team has completed a full round of visitations, the 
participants meet to assess the process. At this session, they 
attempt to identify their successes and failures and/or the par- 
ticular interpersonal relationships which may have facilitated 
or hampered the process. Following this phase, the entire 
sequence of events is repeated. 

Texas Tech University. In the true spirit of formati\'e peer 
review, the program at Texas Tech University' (Skoog 1980) 
"gives priority to data and suggestions that will enhance or 
build on existing patterns of strength in the faculty member s 
teaching repertoire" (p. 23). In practice, it is similar to the 
University of Cincinnati program, though it has distinctive fea- 
tures. The process is carried out by a team of players: the pro- 
fessor who will be obser\'ed, a colleague v^'ho scr\'es as team 
leader, one to three additional faculty members, and, on occd- 
sion, a graduate student. 

The process hiLS five stages; preobseiTation conference. 
obser\'ation, analysis and strategy- session, post-obser\ation 
conference, and post conference analysis. At the pre- 
obse r\'at ion con fc re iic c : 



relatiotiships are shaped, ififonmitkni is shared, af id goals 
are set. . . . The ohsen ve telL; (he team about the session to 
he observed— its objectives, what instrttctiotial modes trill 
he used, I and I what the student role will be (p. 23). 
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At this time, the obser\'ed professor and the team arrive at 
consensus on the aspeas of teaching that will be assessed 
and the procedures that will be used to provide relevant data. 
The procedures may include videotaping and student inter- 
views as well as observation. 

During the obser\'ation stage, the team visits the professor s 
class, gathering the types of information agreed upon earlier. 
Normally, observers spend from 15 to 20 minutes in the 
classroom. 

At the analysis and strategy session, where the observed 
professor is not present, team members attempt to reconstruct 
the events of the classroom visit. In determining its feedback 
strategy, the team compares information provided by the 
instructor at the pre'Obser\'ation conference to what occurred 
during the classroom visit, tiying to describe, not judge, the 
professor s teaching. 

At the post-obser\'ation conference, the obser\^ed professor 
and the team of observers meet to discuss events that 
occurred at each stage of the process. As the team presents 
its findings, the obser\'ed professor "should be involved in 
asking questions and reacting to the team s observations and 
suggestions" (p, 24), 

In post-conference analysis, the process itself is assessed. 
Of particular importance is evaluation of the effectiveness of 
the strategies employed during the posl-obser\-ation confer- 
ence, since that stage is vital to how the professor will use 
the process to improve his or her teaching. 

University of Kentucky College of Medicine. The Depart 
ment of Community Medicine of the University' of Kentuck>' 
College of Medicine s peer review program (Cowen, Davis, 
and Bird 1976) is considerably different from most programs, 
because content of a proposed course first was presented to 
faculty obser\^ers rather than to students. There are two dis 
tinct components of the program. The first, discussed here, 
is limited to obserx^ation; the second, described later, includes 
peer review of course materials as well as obser\'ation. 

Tlie peer obscr\'ation component has, in effect, four stages: 
a professor's presentation of proposed lectures to colleagues; 
general discussion about and written peer evaluations of the 
teaching; a professors self assessment, review of colleagues' 
written comments, and revision of the proposed lectures; and 
the professor s revised presentation to colleagues. In essence. 
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course development becomes a more structured prcKess, no 
longer left entirely to the course instructor s discretion, 

lJ:iiversity of South Carolina, The formati\-e peer evalua- 
tion prcKess at the University of South Carolina (Bell, Dohson, 
and Gram 1977) has not only elements in common with most 
programs— pre obseivation, obser\-ation, post obser\'ation— 
but also two somewhat different components, Tlie program's 
sequence of activities are: an orientation session involving 
all program participants; a pre-obser\'ation conference 
between obser\'ed professor and obser\'er; v^'o classroom 
obserx'ations; post-obser\'ation conferences; and a debriefing 
session. 

The orientation session includes a general discussion 
among participants on their expectations tor the program, 
the identification "of teaching st)'les or characteristics that may 
distract from learning" (p, 16), the generation of suggestions 
for managing classroom discussion, and evaluation of "general 
communicability ' of course content. The debriefing session, 
again im'olving all participants, is essentially program eval- 
uation, but can also include discussion of issues raised in the 
orientation session. 

Evergreen State College. The peer review program at FA'er- 
green State College (Elbow 1980) is different in many respects 
from other programs. In some instances, the differences are 
fundamental; in others, the differences are in emphasis. One 
distinct difference is in the role played by the obser\'er. At 
Evergreen, "one faculty* member each quarter would be freed 
from teaching to be a '[designated] visitor and would spend 
each week visiting a faculty- member who had volunteered 
to be visited" (p. 25). 

The process consists of a series of four events: pre- 
obser\*ation assessment, pre obser\'ation intcr\'iew, obser\'a 
tion, and post obser\-ation conference. Of pre obser\'ation 
assessment. Elbow writes: 

Well before the week oj visitatiofi, I cisked the fcicnll)' 
members whom I was to fobsenv/ to write ifijormaliy about 
what they wanted to work on, the parts of their teaehiti^ 
that pleased or did not please them, the changes that they 
iratited to produce in students through their teaching and. 
more personally the satisfactions arid dissatisfactions that 




came to them from teaching. I aLso invited stories about 
good and bad moments not only as teacher but also as stu- 
dent {\x 26). 

Tlie pre-obsenation inter\'iew between professor and ob- 
sen-er is an opportunity for them to discuss what the facuirs' 
member had wxitten for the pre-obser\*ation assessment and 
to lay groundwork for classroom visits. Elbow is precise in 
indicating what he hoped to accomplish at this point of the 
prcxess: 

First, I listened for statements of goals and probletm so that 
I could see what I was being invited to do and the kind of 
permission that I was being given. I wanted to be saying 
in effect, "You set the agenda for my visits and feedback 
I will give you only the kind of feedback that you desire. 
You are the boss. . . . 

The second thing I looked for in these initial conversations 
was memorabilia, anecdotes and portraits from the person \s 
memory of teaching and of being a student. I wanted to 
hear about good mornents and bad ones, interesting per- 
sonalities who seemed important, incidents that somehow 
stuck in mind. Thus was a powerful way for people to find 
out more about their real goals, not just their professional 
goals. People often wandered into insights as they told me 
incidents that somehow stayed in their minds through the 
years {p, 27). 

Over the course of a week of visitations, the obser\'er is pres- 
ent at virtually all occasions when the professor and students 
interact. The cxrasions include chisses, seminars, and indi- 
vidual conferences, even sessions in which faculty interact 
with others in the absence of siudenus. 

Prior to the post-obser\'ation conference, Elbow obser\'es: 

Before the fiyial long conversation at the end of the week 
(or the beginning of the next I where I brought together my 
most important perceptiotis and made my recommenda 
tions, if I had any I i^siuilly down a couple of times to 
play hack my perceptions of whot bad happened in a 
semirtar, class, or conference. 
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/ took extensiiv notes during the iniiial conversation and 
subsequent observations. At first, I wanted only to aid my 
memory— and perhaps also to cover my nervousness— but 
it turned out to make the process one of mirroring what 
happened— both in the room and in me— not one of reach- 
ing conclusions Also, I found that I had more to say than 
if I sat back to observe and wait for wise insights When I 
left the note-taking machine on full throttle, perceptions, 
reactions, nuances of feeling and et^m metaphors readily 
came lO mind (p. 27). 

Videotape or audiotape recordings and inter\iev.^s of students 
can be used to supplement the essential program compo- 
nents. It is information gathered in a number of ways that is 
the basis for the post-obser\^ation conference. 

University of New York, Cortland. In the master-student 
scheme devised by Shatzkv^ and Silberman at the University 
of New York, Cortland, two faculty^ members— from different 
academic disciplines— collaborate to improve each other's 
teaching ( 1986). Their aims are "to acquaint each ouicr with 
an introductory^ course in a subject in which neither of us had 
any preparation," "to ohser\'e one another's teaching tech- 
niques from the student s point of view/' and "to see what 
problems students have in comprehending the material dis- 
cussed in each class" (p. 119). 

In the role of student, the professor attends each class, com- 
pletes all written assignments and projects, and takes ail tests. 
The professor also attempts to converse informally with stu- 
dents about their understanding of course content and the 
professor s teaching. 

A professor as student can bo helpflil to a colleague, accord- 
ing to Shatzky and Silberman, 

in pointing out problems in presentation, the design of 
assignments, or even the line of questions on an examina 
tion. Skill and experience enable the colleague to diagnose 
and articulate such problems clearly and objectively. Pro- 
vided the instructors trust and respect each other's judgment, 
criticism can he presented constructively— without the inher 
ent ambiguities associated with student evaluatiom or the 
questionable value of one- time peer visitation and evalua- 
tion {p. 119). 




New York University, In an attempt to learn more about 
the nature of teaching aiid learning and to improve their stu- 
dents'—and their own— performance, two composition 
instructors (Rorschach and Whitney 1986) describe a forma- 
tive peer evaluation program they de\'eloped at New York Uni- 
versit\-. TheN* explain the participant obserx^er method as 
follows: 

For 15 weeks, we attended each other's freshman writing 
coarse, which met twice a week for a total of three hours. 
. . . The teacher in each class taught as she normally would, 
while the observer took the role of a student, participating 
in class discussions, uriting drafts for most of the assign- 
ments, and sharing his uriting in peer groups uith the other 
students \Xe each kept a notebook on the experience, and 
we met once a week for about an hour to discuss what had 
been happening {p. 160). 

Later in the term, when obser\*ation alone failed to provide 
enough informavion about differences in student attitucles 
and behaviors to satisfy the instructors, they also report using 
t^^o additional data-gathering methods. The\^ explain that tape 
recording of classes was chosen "so that we could look more 
closely at our classroom beha\'ior. Though we shared a sense 
that we were behaving differently in our roles as teacher, it 
was not immediately apparent how" (p. 163). And they report 
examining each others lesson plans "to see if any differences 
in the progression of each course would help us explain the 
experienced differences in the [class culture) we had pro- 
duced** (p. 163). 

Rorschach and Wl^itney also report discovering how to 
develop a ckussroom culture in which students share authority 
with the teacher and how students can be helped to develop 
a sense of autononiy its v^Titers. Tlie authors also are con 
vinccd that their participant c)hser\'er niethod has implications 
for changing the nature of the teaching and learning prcK*ess 
for both students and teache*- They write: 

The situation that we found ourselves in is not uncommon. 
Most teachers experience ct yawning ^i^ap between the 
abstractions about education presented to them by univer 
sit}' researchers and thcprcssinj^ decisi(nis about what to 
do in their classrooms tomorrow morning. Perhaps the fault 
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for this gap lies neither with the researchers nor with the 
teachers, but with the situation: the isolation of the one- 
teacher classroom, rigid scheduling patterns, limited or non- 
existent opportunities for ofigoing collaborative iiiqtwy, and 
the lack of sufficient precedent and support for carrying 
out such inquiry even when the opportunities for U could 
be made. The great bulk of useful human knotvledge, after 
all is probably geyierated outside of laboratories and librar- 
ies by groups of people working to solve common problems, 
talking and thinking together as they go. Through such col- 
laborative inquiry we teachers can become researchers in 
our classrooms and tum our valuable classroom experience 
into useful knowledge for ourselves and for one another 
(pp. 171-72). 

University of Birmingham. The "course ev^aluation 
scheme" at the University of Birmingham, England (Mathias 
and Rutherford 1982a, 1982b), involves the gathering of data 
from both peers and students, and from a variet\' of methods, 
of which observation is but one. According to Mathias and 
Rutherford, the purpose of the program is "to help [faculty] 
to find out how their courses were being received by students, 
to identify areas of difficult)' and to explore w^hether students 
understand and could work for the aims [facult\') had in 
mind" (1982b, p. 48). 
Tlie essential elements in the prcKess are as follows: 

A common sequence of events would if waive the evaluator 
in the observatiofi of a fairly self contained 'episode' of the 
course, usually centered arou?id a particular topic. For 
example, the evaluator would attend sevt>ral lectures, tutor- 
ials, practical classes or seminars as an observer. He would 
then discuss the course with o?ie or more small groups of 
students employing a semi structured interview technique 
with the assurance that their anonymity uvuld be presenx^d. 
The evaluator and lecturer would then develop a questi- 
onnaire to follow up some of the maifi issues which had 
emerged, as well as probing other aspects of the course which 
had not been previously investigated. . . . This ([uestionnaire 
was admi)iistered on a voluntary basis to all the students 
taking the coto'se and analysed and interfjreted by the eval- 
uator and lecturer A fifial report was prepared by the lec- 
turer i?i conjtinction with the evaluator. This report, which 
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ckKscribed the procedures and outcomes of the evaluation, 
was presented by the lecturer for dLscussion at one of the 
regular course evaluation ?neetings {l9S2h. p. 49). 

Evaluiitors come from fields of study related to that of the 
obserwd faculty member, though not exactly the same or 
clearly unrelated fields. FAaluators are not selected from totally 
different disciplines because "experience showed that it was 
important for the e\aluator to possess some familiarity with 
the sui^.ject matter of the course if he was to appreciate the 
course activities and the problems that arose from them" 
( 1982a, p. 264). But evaluators are not selected from the same 
discipline either, "to avoid the potential embarrassment of 
exposing difficult problems to a close colleague" ( 1982b, 
p. ^i9). ^ 

California State University, Sacramento hi the Profes 
sors Peer Coaching Program of Cal State, Sacramento, the peer 
coach s primar\' responsibility is to facilitate his or her col- 
league s .self assessment of, and self reflection on, teaching. 
•Ilrai role is not to exaluate, direct, prescribe, or even suggest 
changes. Rather, it is to dcx:ument what is taking place in class 
and to conjoin the teacher in conversation comparing what 
is actualK- occairring to what the teacher believes or hopes 
is happening. 

In effect, the coach is a mediator betw^een the realities ot 
the teaching e\ ents and the teacher's perceptions of these 
same c^vents. Through these conversations, teachers reportedly 
change "with greater punx)seful innovation in teaching than 
CN-aluation provided by others'* (Stoner and Martin 1993, 

In this intensive, yearlong program, voluntaiy participants 
( reciuiring tacult\* to participate is expressly proscribed), 

attend hi two hour workshops (seven each semester) to 
develop specific coaching skills, and talk with other partic 
ipants about coaching and teaching, . . . During weeks 
between sermnars, participatits spend time coaching. A 
''coaching cycle" is completed when a participant has been 
ctuu hed m a pre teaching conference, observed by the 
coach, and coached in a post observation conferetice and 
has performed the same functi(msfor the partner 
(pp. H^)). 
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When participants meet in pre conference to talk about the 
forthcoming classroom visitation, the coach employs "specific 
communication skills in questioning, probing, paraphrasing, 
and decoding nonverbal messages to [elicit from the teacherl 
precise descriptions of the lesson to be taught" (pp. 9-10). 
During this session, the peer coach engages the teacher in 
a discussion of goals and objectives, anticipated teaching 
strategies, the means by which student learning will be ascer- 
tained, and the methods and procedures that will be 
employed to document what takes place during obser\'ation. 

A critical stage of the program is post-obser\^ation coaching. 
In these sessions. 

C 'sing specific coaching skills, particularly questioning and 
paraphrasing the coach will assist the teacher in recalling 
the teacher's personal assessment of the class, the teacher's 
behaviors and decisions, and the students' behaviors, and 
assist the teacher in making reasoned inferences about rela- 
tionships between student achievement and the teacher's 
thinking and actions. It is here that the coach will supply, 
if asked, to the teacher the specific data collected by the 
coach {p. II). 

Pre-()bser\-ation and post -obser\'at ion coaching protocols arc 
provided by Stoner and Martin in their text as Figures I and 
2, respectively. 

in that the program s (Objective is to encourage the teacher 
to assess his or her teaching accurately by comparing, with 
a coach's assistance, reality with perception, the Professors* 
Peer Coaching Program is considerably difTereni from more 
directive formative peer evaluation programs. And, in that it 
explicit!)' empowers teachers to make changes in their teach- 
ing, the program promotes regular self-assessment. This is 
in marked contrast to episodic evaluations by others. 

Videotaping of classes 

llie use of videotaping of classes to improv e teaching is not 
apparently vciy common in higher education. Only one pro 
gram of formative peer rex'iew in which videotaping was the 
central element is described in recent literature on instruc 
lional development. Othenvise, videotaping has been 
employed as an adjunct to other methods of formative 



evaluation. 

Tliere are a number of reasons that could explain why 
videotaping is neglected in the process of instructional 
improvement. First, many faculty- are nc)t adept at using the 
technology'. Second, they are not willing to take the time to 
use it. lliird. they feel threatened by having a permanent 
record made of their teaching. Fourth, they see videotaping 
as an alternative to classroom obsen^ation, of little value in 
its own right. 

Universky of Hartford. The model developed by McDaniel 
at the Liniversity of Hartford has three stages. In the first stage, 
facultA' collaborate in establishing standards of effective teach- 
ing that apply "to their particular setting, purpose, and student 
population" '( IQH'", p. 98), 

In the second stage, participants* clashes are videotaped 
on two occasions. During video playback feedback they, with 
the help of a teaching consultant, evaluate their performance 
according to the standards of effective practice they have been 
a part c)f developing. Tliis process involves, among less formal 
procedures, the "stiniulated recall ijiterview'' (Craig, Redfield, 
and Galluzzo 1986), where specific episodes of teaching are 
pla\'cd back and analyzed by the teacher and the consultant. 

At the fii-^al stage, faculty have the opportunity to view sue 
cessfiil teaching and to discuss how the methods demc)nstrate 
staiuiards of effective teaching they established earlier. Faculty^ 
"find obsening others on videotape is as beneficial as watch- 
ing themselves" (p, 99). 

Evaluation of course tnaterials and assessment of 
instructor evaluations of the academic work of 
students 

Peer review of tiie course materials of colleagues and of 
instructor eva!uatic)ns of the academic work of students are 
methods that remain largely unexplored, W^iere used, these 
methods are employed nivxstly as adjuncts to classroom obser 
vation. not as integral elements in the process of instructional 
improvement. More prominent roles for these methc)ds may 
come about if the teaching portfolio is used expressly for the 
juupose of impro\*ing teaching, 

San Jose State University. '^Iiile more exhortative than de 
scriptive. Galm nonetheless tells us some things about how 
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formative peer review of course materials can be helpful to 
faculty in post-tenure evaluation ( 1985). Galm's discussion 
is general rather than precise in detailing the mechanics of 
the process. 

At San Jose State, t%vo of five peer re\'ie\v sessions are 
devoted to evaluation of course s-N'Uabi. In these sessions, pro- 
fessors discuss the \va>- in whicli they present courses to stu- 
dents— rationale, requirements, and grading criteria— and the 
methods and materials that are used to present the course 
content. The remaining sessions are devoted to technolog\', 
classroom discussion techniques, and course content. 

University of Kentucky College of Medicine. Besides 
the obseration prcx-'ess described earlier, the llniversity of 
Kentucky' program (Cowen, Davis, and Bird 1976) also 
includes peer review ot course materials combined with cla,ss- 
rooni observatioii. The course materials-classroom obsen-ation 
program is associated w^th the de\'elopment of new seminars. 

In this program, a faculty member preparing to teach a 
seminar for the first time provides colleagues witli materials 
he or she plans to use. The colleagues "prepare the material 
as though they were students" (p. 130). After completing the 
"assignments," the colleagues play the role of students while 
the faculty member Jirect.s the seminar. According to the 
authors (p. 131), "it was hoped this mechanism would 
accomplish two goals— peer review of the seminar itself and 
the education of the faculty in the content and techniques 
of the seminar presentations." 

Fairleigh Dickinson University. Actual teaching portfolio 
entries illustrate how peer review of course materials and of 
instructor e\'aluaiions of students' academic work could be 
used to improve teaching. Tliese documents display not just 
"the final products of teaching but its processes— the thoughts 
behind the actions — they also re\ eal much about teaching 
to colleagues involved in the development and review of port 
folios. . . . Occasions where facult)' examine one another \s 
portfolios could be occasions for cultivating new and richer 
ways of thinking about inquiry- into the scholarship of teach 
ing** (Wgerton, liutchings, and Quinlan 199 L p. 6). 

'I\\'o portfolio entries of a I'airleigh Dickinson management 
professor (Ottaway 1991 ) are a course syllabus and the 
teacher's rellections "on the pedagogical thcoiy undergirding 




thai syllabus and on how a changing mix of siudents will 
affect the next version of the course" (Edgerion. Huiciiings, 
and Quinlan 1091. p. 19). Ouaway obseives. among other 
things, that a class composed of increasing numbers of full- 
tirpe. traditional age undergraduates requires different teaciv 
ing approaches than a class pre\'iousl\- composed mosth' of 
pan lime wxnking adults. The retleciive essay also describes 
how he proposes to accommodate and challenge boih groups 
of students. Otiaway's colleagues could probably leani from 
his portfolio enu ies. and he from theirs. 

Harvard College. Porifolio entries of a Han ard hisioiy pro- 
fessor (Wilkinson 1991 ) include an undergraduate's book 
review, the professor's feedback lo the student, and the j^ro 
lessor s relleciions on the comments made to the student and 
on the grade assigned to that piece of work. The author also 
provides insights into how praise and constructive criticism 
can be used to motivate students. 

Multi-Institution Programs 

Lilly Endoivment Teaching Fellows Program 

Since 10" i. the Lilly Hndowment has spcuisored the Teaching 
Fellows Pix)gram. formati\ e peer re\iew projects for junior, 
nonienured faculi\- at sewral major research uni\ersities. I ni 
versiiies are eligible to aj^ply for Lilly l-ndowment funding 
lo support a project lor up to three yeai\s. The discussion pre 
.seined at this point, then, is of a genre of projects rather than 
an\- one program example. AcctM'ding to Au.stin. a program 
iK)rmally: 

i}ii't)livs six /f> /tv/ touivc sfivani ji/fiior facidlr nu'fnbcrs 
U'hi) (OV af)f)(n'NtC(/ (is fclloirsfor a one year tcnn. I siially, 
Jclknrs hcu v been (it the histitittion hctuvoi one and fiiv 
wars. ftiLidty in their fint year arc ^oicraliy not eligible, 
on the assumptioii that they are busy with initial accliniation 
to their }}en ein irotnnentsi 1902a. p. "" t ). 

Programs ha\ e had a number of common elements and their 
own distin.uuishing features, .-Vs Au.stin explains 

.1 nniivrsity's I illy Te(U hni\^ /'elloies I'ro^nuu tyfneally 
nn-oli es fortnightly meetings of felloirs to discuss teaching, 
indii Kinal projects f tensed on tcachinjif. some released time 
frotn nsnal course res/nmsibilitics, and. often, senior faculty 
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mentors. The endowment also spofisors fall arid spring week- 
end conferences on teachhig, held at different sites through- 
out the eastern half of the country, where fellows meet their 
counterparts from other universities and interact with 
nationally respected contributors to knoivledge about teach- 
ing. . . . Within this general pattern, each institution shapes 
the program to its own needs ( 1992b, p. 88). 

Readers who want more information about program compo- 
nents should consult Austin 1992a, pp. 

In many Lilly Teaching Fellows Program projects, mentoring 
is a key compone^^ Austin notes that: 

While a i^ariety of mentor arrangeme?its have been suc- 
cessful in teaching fellows progra7ns, one factor associated 
with success is eviderit: flexibility in approach In flexible 
programs, fellows choose mentors on the basis of mutual 
interests and goals, regardless of the potential mentor's 
departmental affiliation. While the program directors sug- 
gest ways in which fellows and mentors might choose to 
interact and activities that they might wish to share, the spe- 
cific dimensiofis of the relationships are left to each mentor- 
fellow pair. If a mentor turns out to be less helpful than 
expected, program directors who take a flexible approach 
recommend that the fellow simply add a second, more com 
patihle mentor while still keeping the first ( 1992a, p. ""8). 

A number of conclusions Austin has reached from her cjual 
itative study of ihe Teaching Fellows Program are reported 
in the section on Personal and histitutional Benefits. Lending 
credence to her findings is the examination of projects 
extending over a period of 15 years, including inten iew 
data— opinions of recent fellows and reflections of senior 
faculty who took part in a project year> ago. 

Partners in Learning. Partners in Ix*arning (PIL) is New 
Jersey \s statewide peer obsen*ation .student inteiview iiistruc 
tional development program for po.stsecondar\* faculty. Cur 
renily under the auspices of Mie New Jersey Institute for Col 
legiate Teaching and Irarr. g (NjiCrD, it is based on the 
conceptual model of Kiitz and Henr\' ( 1988). described earlier 
in this .section as the University of New York, .stony Brook, 
program. Tliis voIuntar>\ "emphatically nonjudgmental" pro 
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gram has evolved during its shoa life froni what its creator, 
Joseph Katz, enx isioned to its present form. It now invoK'es 
having faculty: "pair themselves off, preferably with someone 
from a different discipline" to observe each other's teaching; 
interview students about what and how they are learning; 
meet as a pair to dii <:uss what is happening in the classes 
being obser\'ed and what students arc saying in the inter- 
views; meet as a group with other pairs to discuss common 
and unique experiences; and wite reflective essays on what 
they have learned about teaching and learning from the class- 
room observations, student interviews, aiid collaboration with 
colleagues. 

Tlie process of peer classroom obser\^ation, student inter- 
vie^'^s, and collaboration takes place over the course of, 
ideally, two terms of instruction. During the first term, one 
member of each pair obser\^es one of the other's classes, once 
a week if possible or at least biweekly. During the following 
term, these roles arc reversed. Like the program manual indi- 
cates, "the success of the process is greatest if the obser\'er 
becomes an accepted part of the class (essentially invisible). 
To achieve this invisibility, the obser\^er should begin early 
in the semester, and the observ^er should attend regularly" 
( NJICa 1991. p. 8). While attending classes, 

the oksenvrpays dose attention to the process and dynamics 
of the ^roup. The ohsen^er notes the effect on the students 
of the instructor's presentation, the manner and types of 
questions generated, interactions betiveen students, group 
discussions, non rerbai com municat ion ^ei^ery thing that 
happens during the class (NJICYI 1991, p. 9). 

In PIL. the student interview is indispensable, since a "wealth 
of information and insight" about teaching and student learn- 
iiig is obtained in this way Each participant— teacher and 
observer- intcn-iews three students on several cKxasions 
( usually three to five times) each semester, focusing initially 
on rather general topics but eventu;^lly on more substantive 
matters relative to student learning. inter\'iewers are enccuir 
aged to select students representing ""a broad section of the 
class in age. gender, race, classroom demeanor, or academic 
performance" (N[lICTn>, IWl, [v 10) and to employ a flexible, 
senii .structured approach. (Recommended inter\'iew protcK* 
ols are provided by XllCn^ 1 1991, pp. 10 12) and by Wool 
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wine [1988, p. 49].) 

Faculty pairs meet frequently (ever\' week or two) lo dis 
cuss what has been taking place in the classes obser\'ed and 
what students are talking about in the inter\'iew^s. Hie critical 
initial meeting of each pair should occur prior to obseivations 
and inter\'iews, for it is during this occasion that "an imme- 
diate intimacy' between the partners [can develop, and an atte- 
nuation of] tension that either might be feeling about the 
obser\^ation process [can ocxur]" (NFIICTL 1991, p. 10). 

About once per month, all pairs on campus meet to con- 
sider topics of common interest. These topics include what 
is happening in the classes being c)bser\'ed and in the student 
inter\dews, problems encountered, teaching theories, learning 
st>'les, critical thinking, recent research on pedagogical issues, 
and the like. 

At the end of each term, both participants write essays 
where "the focus may range from thoughts about classroom 
obscr\'ation, student inter\'iews, and interaction with col- 
leagues to any facet of her own teaching that these activities 
may have led the participant to examine" (NJICTL 1991, p. 
14). NX^ere faculty give permission, their essays, or exceq")ts 
from these pieces, could be distributed across campus, alk)w 
ing ideas and insights about teaching and student learning 
to be shared wnth colleagues. 

Program coordinators, participanLs, and evaluators tell us 
not only that PIL works but also why it vs orks and how it 
improves teaching and student learning. Steve Golin, Katx/s 
successor as state program director, explains why it has been 
successful: 

// uvT'ks because the program ongoing, because it is 
decentralized, because it /5 faculty owned, auci because the 
process //5cV/aj transforming. . . . Facult}* who ohsenvauc/ 
are observed, who iuterview studetits, who meet with a 
partner, are engaged iu an ongoing process. . . . Feedback 
from. oi4r students and our coileague is continuous, in 
response, we tr}\*;onie new things, and we get feedback on 
them. . . . The facidt}' pair is lafgely autonomous, it charts 
its own directions. . . . The pair shapes its own rersion of 
the process. . . . Voy quickly, facuit}' ciaun ownershif). . . . 
faculty respond with real creatirit}' and initiatire to a pro 
gram that theyperceire is not only for them but by them. 
Collaboratifig with a peer is itself transforming. . . . For 
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many/aadi}) the student inteniews are civu more pow- 
erful for our selj-transjortnation {GoWvi 1990, pp. 9-10). 

How PIL helps professors improve their leaching ;ind their 
students' learning is at least as important as knowing why. 
Tlie value of the program is addressed later, in the section 
on Personal and Institutional Benefits, under the headings 
Improvement in Teaching and hnprovement in Student 
Learning. 

From idea to prototype 

As we complete the v^Titing of this book, the American Asso- 
ciation for Higher Education, in cooperation with Stanford 
Universit>\ is initiating a multi-institution study titled "From 
Idea to Prototype: The Peer Review of Teaching." Coordinated 
by Lee Shulman, researchers at the 14 participating institutions 
(all reseaich and dcx'toral universities except one) are "con- 
ducting small-scale experiments in peer review exercises 
intended to reveal the 'pediigogical thinking behind various 
aspects of teaching practice [and exploring] a variety of strate 
gies in being colleagues to one another. in teaching as they 
are in research" (American Association for Higher Education 
1993, p. 18). 

Assessment of Program Strengths and Weaknesses 

To this point, we have described, in a rather detached manner, 
several programs of formative peer evaluation of teaching. 
Chronicling the developnieiit of these programs is important, 
we believe, because it demonstrates that increasingly sophis- 
ticated programs, ultimately more successful in affecting better 
teaching, evolve - rather than appear full fledged--and will, 
no doubt, continue to develop. 

Some formative e\*aluation programs, particularly those put 
into place during the 19*"0s luul early 1980s, clearh' are pro 
tot\pes, quite limited in scope and scale. Others, especially 
thcxse implemented in recent years, are more fiilly dewloped, 
more comprehensi\-e in scope, and more ambitious in scale- 
two even multi institutional. Some of these programs, or ele 
menis of them, almost certainly will have greater luid longer 
lasting impact on improving leaching and student learning 
than others. NXe examine now dift'crences among these pro 
grams, noting what we consider tl^nr salient strengths, greatest 
weaknesses, and critical limitaticMis. We focus much of our 
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attention on peer observation, since this is a common element 
among most of the programs, the only one that invites sig- 
nificant conipariscns. 

Most programs . nvolve collaboration between two profes- 
sors, one being ot.served by the other. In all colleges and uni- 
xersities where pa rs work together except Evergreen, the 
roles are reversed at some point, so that classes of both par- 
ticipants are observed by the other. We consider this reciprcx'- 
it>' a strength becajse it encourages intelligent professional 
people to learn from each other. Such an aiTangenient may 
benefit one participant more than the other if one teacher 
is more experiencxl or competent thiin the other, but a 
mentor mentee re ationship can be mutually beneficial as 
well. 

At Evergreen, by contract, one faculrv- member (the desig- 
nated obser\'er) visits classe,s of different colleagues each 
week. There is merit lo this OTangement, too, since he or she 
can attend to this 'ole for an extended period, developing 
a systematic process for obser\'ation over time. It is somewhat 
limiting in the sense that fewer faculty' learn from each other's 
strengths and we: knesses. 

Other programs involve tv^'o or more obser\'ers. At Cincin- 
nati, three professors collaborate in the reciprocal arrange- 
ment described earlier. Its strength lies in having a teacher 
receive feedback on his or her teaching from two colleagues. 
A pinential diffic ulty lies in scheduling visitations and con- 
ferences. At Tex:Ls lech, a team of obser\'ers visits a colleague's 
classes. The tea n approach is attractive in obvious ways, but 
disadvantages exist as well. First, it is a "single player," rather 
than reciproca'. plan, limiting because only one team 
member s classes are critiqued. Second, scheduling can be 
unwieldy, sin :e several faculty are invoK'ed. 

Among the most controversial issues with which program 
dex-elopers must reckon is how to pair up the participants. 
In the programs we've described, teachers and ohser\'ers have 
come from the same field of study ( New York Uni\'ersity and 
San Jose State), from related disciplines (University of Ken 
tuck-)- and l'ni\'crsit\' of Birmingham, England ), and from unre 
lated disciplines (Cal State, Partners in Ijearning, and srNTi' 
at Portland and Stony Brook). \Xe have suggested self 
selection of collaborative units, a prcK*e,s.s that can be guided 
by participants' objectives and needs. We've made Further 
()bser\'ations about and recommendations regarding this issue 
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in the section on MethcKis of Formative Evaluation, under the 
subhead Direct Classroom Obsen'ation. 

The number of classroom visits has ranged from two per 
term (South Carolina and Cincinnati) to weekly or biweekly 
(at Cal State, Sacramento, and SlINY, Stony Brook, and in 
Partners in Learning) to ever>' class meeting (Evergreen, New- 
York University', and SUNY, Cortland). Clearly, two \'isits is 
inadequate for developing close facilitative relationships and 
for helping colleagues affect significant changes in their teach- 
ing. Obviously, more fi'equent visitations can be beneficial 
but require a greater time commitment from participants- 
essential, we believe, if the programs are to be valuable to 
faculty' and students. We recommend that visits occur as often 
as fe-asible, but certainly no less often than ever\' other week. 

In all programs except Texas Tech's, obser\'ers attend a Rill 
period of a colleague's class. Since there are many aspects 
of teaching that can be critiqued by colleagues, we are con- 
vinced that peer re\'iewers should remain in a classroom for 
entire class periods. We also believe that to do otherwise 
wx)uld disaipt classroom dynamics unnecessarily, even more 
than visitation does by its very nature. 

At SIW, Stony Brook, and the University of Birmingham, 
England, and in Partners in Learning, faculty interview 
selected students on what and how they are learning and 
obser\*e colleagues classes. We believe the student inteniew 
can be a potent force in improving teaching und student learn 
ing. because it invites a more comprehensive review, pro- 
motes the bridging of faculty and student cultures, and fosters 
the creation (or reinforcement) of a campus ethos in which 
teaching and student learning are valued. If included in for 
mativc peer evaluation, inteivicws should he conducted reg 
ularly by both teacher and obseiver. Yet, we realize that the 
student inter\-iew. like videotaping of classes or other 
meiliods, may not work for everyone, since some faculty may 
feel so uncomfortable in certain situations that the result 
would be counterproductive. Generally, formative peer eral 
uation should be tailored to the individual n(*eds of 
participants. 

An ' >hserver usualK' sits unobtaisivoly during the classroom 
visitation. In New ^btk rniversily s program, however, he or 
she is participant oirseiver. Tliis latter arrangement can work, 
wc lK'lie\ e, if ihe ohserwr is accepted as a regular class 
member. There is the danger, though, that he or she might 
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be viewed as "star student/' a situation in which the natural- 
ness of the classroom environment would be disturbed. In 
most instances, we suggest that the peer reviewer be a silent 
obser\'er, obser\'ing process and result and then pun-eying 
that information to the teacher. 

Tlie process by which feedback from obser\'aiions is com- 
municated to the teacher varies somewhat from program to 
program. As envisioned by Katz and Henry (SLINY, Stony 
Brook), it is relatively structured. In other programs, such as 
those as San Jose State and SUNY, Cortland, the process is 
quite informal. At Cal State, Sacramento, the obser\'er s role 
is facilitative and nondirective. Generally, we believe that feed- 
back should occur in a semistructured emlronment, guided 
by flexible protocols like those recommended by Stoner and 
Martin ( 1993) and Katz and Henr>' ( 1988, pp. 1 12-25). 

Some programs include occasions where all program par- 
ticipants on a campus meet to discuss common problems and 
issues on teaching and learning. VC^iile requiring even more 
of panicipants' time, such meetings can be intellectualh' 
engaging and a source of motivation for facults*. We recom- 
mend scheduling an orientation meeting right before :he tf^rni 
begins and tu^o additional meetings during the semester. Such 
(xriusions might include presentations or discussions led by 
experts in teaching and learning, although extreme care must 
be exercised in selecting speakers to be sure that what they 
niay have to say is relevant and worthwhile. 

In all but three of the classroom observation programs 
we've described, other methods are "add-ons," not integral 
components. At SLW, Stony Brook, and in Partners in Learn 
ing, the student inter\*iew is an integral element. And at Ever 
green, an intensive pre-conference inter\'iew (between 
teacher and designated obser\-er) is a vital element. It is in 
the lack of comprehensiveness that we find most programs 
wanting. None of the programs include e\'aluation of course 
materials or assessment of instmctor e\'aluations of the aca 
demic work of students as any more than an adjunct to obser 
vation. We are convinced that these methods should become 
essential components of formative peer review prtK*ess, exam 
ined either as separate entities or ^ts part of teaching portfolio 
anaK'sis. In either case, a SA\stematic procedure for assessing 
these products should be developed. 
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Common Elements 

The programs of peer observation examined in this section 
have a number of common elements. Tnese elements are 
offered at this point to guide practice. 

1. Programs should be built on the premise that "good 
teachers can become better'' (Carroll and Tyson 1981 ); 
programs should not be considered remedial 

2. Faculty participation should be voluntarv'. 

3. The ohser\ed teacher and the obser\'er should be trusted 
and respected by each other. 

4. Classroom \ isits should be recipixxal (a faculty member 
should he. in turn, obser\'ed and ohserxer). (Only Ever 
green has a designated faculty obser\'er.) 
Obser\atioas should c^ccur by inviiaiion onK' (there 
should be no suiprise \'isits). 

6. Participants should determine in advance ^\'hat aspects 
of teaching are to be assessed. 

Participants should also determine in advance what other 
procedures, if any, are to be employed in assessing 
perfonnance. 

H. *rhe lines of communication between the obser\'ed faculty 
men^ber and ilie obserxer should be open (feedback 
should be both candid and tactful ). 

9. A balance between praise and constructive criticism should 
guide the feedback process. 

10. Results should be kept strictly confidential and apart from 
summative evaluation- 
Many of these guidelines also could apply U) the use of video 
laping of classes, evaluation of course materials, assessment 
of instructor evaluations of the academic work of students, 
and analysis of teaching portfolios. 

Evaluation of Programs 

In all instances, authors of programs described in this section 
report that faculty members believed their teaching had 
improwd as a result of feedback provided colleagues. 
However, evaluation of programs appears to be limited to sell 
reiK)rts of participants: at lea.st, no information is available 
suggesting that mcne rigorous evaluations were conducted. 

In two empirical studies of faculty development programs, 
modest, though stati.stically significant, improvements in stu 



(Afllatxffvthv /Vtr A'c/vV//' 



102 



dent evaluations of faculty and in student learning are sug- 
gested (Erickson and Erickson 1979; Hoyt and Howard 1978). 
Whether the results of these studies can be extrapolated to 
formative peer review is not known. It is clear, however, that 
s>^stematic studies of colleague evaluation should be 
undertaken. 

No method of formative peer review of teaching has been 
employed widely in higher education. In sections that foll()w^ 
we look at factors which may detract from faculty' members' 
willingness to participate, what might encourage them to do 
so, and how programs of instructional improvement of this 
type can be valuable to facult\\ students, and colleges and 
uni\ersities. 
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DISINCENTIVES 

On Us face, it would seem that peer iin'oHemciit in the 
improvement of teaching is a commendable idea. And it also 
would appear that the methods discussed in the pre\ious sec- 
tion, used in combination, should be embraced for the bet 
terment of the academy. ^X'e knov.-. however, that use of these 
methods by peers for formative evaluation has been negli 
gible. A number of reasons ha\-e been cited for the unwilling- 
ness of faculty members to implement and participate in this 
kind of activity. Some of the reasons are based upon practical 
consideraik)ns; others are derived from more philosophical 
concerns. This section examines four of the salient factors 
that we see as disincentives to facuky participation in forma- 
ti\'e evaluatk)n of teaching: acadennic freedom issues; repre 
sentativencss, accuracy, and typicality; subjectivity; and time. 
tacult\- values, and instituti(Mial incenti\'es and rewards. 

Academic Freedom 

Se\-eral .scholars have considered the issue of academic tree- 
dom relative to e\'aluation conducted by direct classroom, 
observation (Cross 1986; Eckert 19S0; Edgerton, Hutchings, 
andQuinlan 1991; Edwards 19^4; Farmer Ur6; Hart 198"'; 
Mauksch 1980; Ropcw Deal, and Dornbusch 19':^6; Few 1992; 
Sweeney and Gntsha 19"^9). ^Iiile acknowledging that there 
is often tlie perception that direct classrocMii oKseivation by 
peers, administrators, or anyone else vic:>latcs a professofs 
academic freedom to teach, scholars have generally concluded 
that academic freedom is not compromised by classroom 
obseivation, because the right of faculty members to deter 
mine what is taught is not circumscribed by the process ot 
ohserxation. That argument is articulated well by Mauksch: 

/ ndcr the mantle of ciaiclemic freedom, teaching is a 
secluded aetirit}' while research, also concerned with deeply 
felt issues of academic freedom, is acknowledged as an 
accountable and challengeable actii'it)\ properly subject to 
scrutiny and checks W hile challenging the methodolog}' and 
techni(fues of the researcher does not threaten his or her 
ri^ht to pursue inquiiy the presence of a risitor in the class- 
room is felt as limiting the teacher's right to choose teaching 
amtoa afid teachhig process. Although there may be some 
h^ h that those in power will wield if appropriate influence 
oi vr either teaching or research, the mere presence of a col 
league, in and of itself is no threat to the freedom of teach 
ing{\m\ p. SO). 
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This undoubtedly is a two-edged sword. On one side, the 
presence in the chtssroom of a peer can evoke a self-conscious 
reflection not only on the manner in which something is 
taught, but on the very content itself— an opportunity for 
improvement in itself On the other side, this wide curricular 
latitude permitted faculty members, while usually considered 
a great strength of American higher education, also allows 
room for t riticism ftom peers who may not share views or 
approaches. 

Academic freedom to teach is a time-honored tradition 
among faculty, but it also is a self-interpreted, self-imposed, 
and largely self-regulated practice except as circumscribed 
U the courts (Poch 1993)^ Consequently, the distinction 
between legitimate concern over free intellectual expression 
and a nebulous fear or distaist of any intrusion can become 
blurred, contributing to the peipetuation of uncritical, clois- 
tered attitudes that may undermine attempts at the improve- 
ment of teaching. These realities have led scholars to agree 
that gaining the support of faculty- in a proc^ess of peer review 
involves overcoming the perceived threat to academic free 
dom to teach. We argue that, in that process, it also may 
require the deconstruction of individual and collective con- 
ceptions of academic freedom in order to lay bare premises 
based upon fear or self interest rather than those that may con- 
tribute to a larger academic integrity. 

As early as 1950, Eckert, while advocating the use of direct 
classroom obser\^ation involving colleagues as evaluators, 
warns that it should not be put into place "without making 
^exceedingly careful advance preparations" (p. 67). This 
advanced planning includes eliciting faculty support so that 
classroom obser\'ation is not seen as an infringement on their 
academic freedom. 

Representativeness, Accuracy, and Typicality 

A second disincentive involves arguments that methods ordi- 
narily used to evaluate teachers are not adequate for assessing 
the dill range of their competencies. Complaints of this t>pe 
usually are aired in connection with direct classroom obser- 
vati(^n, but some of them also could apply to videouiping of 
cla.sses, evaluation of course materials, a,ssessment of instruc 
lor evaluations of the academic work of students, and analysis 
of teaching portfolios, if these methods were used independ 
entl\- of, rather than in conjunction with, other methods. Five 
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common complaints are discussed at this point. 

First, classroom visitation is not sufficient for assessing all 
faculty competencies (Braskamp 19*^8; McKeachie 1986; Sto- 
dolsk\' 1984). Obviously, obseiration should be used only 
for evaluating what cxrcurs when a teacher and students inte 
ract, not for what cxrurs prior to and following deliver)' of 
instruction, processes that also are vital to succe.ssfiil teaching. 
Different methods are required for evaluating these other 
competencies. For that rea.son, scholars almost invariably 
agree that several methods of e\'aluation. used in combination, 
are necessaiy if the full range of teacher competencies is to 
be asse.s.sed (Aleamoni 1981; .\rde 1989; Arreola 1984; Black 
burn and Clark 1975; Bradenburg, ..^askamp. and Or\' 1979; 
Braskamp 19''8; Dre.ssel 1976; Greenwood and Ramagli 1980; 
McKeachie 1986; Romberg 1985; Sauter and Walker 19'^6; 
Schneider 197S- Scriven 1980. 1983. 1985; Seldin 1984; Smith 
et al. 1988; Sodcrberg 1986; Spaights and Bridges 1986; Stev 
ens and Aleamoni 1985; Swanson and Sis.son 19"^ 1 ). 

In adamantly objecting to using direct chussroom obser 
vat ion as the .sole nunhod for lussessing teacher performance. 
Stodolsky insists that it is: 

uftlihely to he fair hecaiL^c cmygitvn obsenation will not 
be nprescntatii v of the range of teaching behaviors used 
by a [teacher j. Erali tutors arc mistaken if they assume they 
are obsening topical behaviors of '{teachers] with the usual 
procedure {\9S-^. p. n). 

But she concedes that '\)ne might u.se obsePv'ation as one type 
of information in conjunction with other materials that could 
provide a more rounded assessment of a teacher" (p. D. 

Classroom obser\'ation is mcxst \'aluabie to teachers. Sto- 
dolsky believes, when it is used for instructional im 
}>rovemeni: 

In formative evaluation, direct obsen'atio?i maybe vcfy 
appropriate if too much is not made of a?iy given obser 
vatiou. Direct observatiOfLs can provide useful occasions 
for dialogues with supervLsors and colleagues Specific occa- 
sions are what teaching is all about, and may provide a very 
appropriate focus for discussing improvenwut. Discussions 
and suggestions that follow observation of a teacher may 
he even more helpftd if it is recognized that he ors/je might 
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teach differently in different situations. Rather than assum- 
ing that one knows a teacher well after a limited set of 
ohsen^ations, one might rather acknoivledge the incomplete- 
ness of that knoirledge (p. 17). 

Second, even where it is employed, critics object to the usual 
practice of sending an obser\-er or tv^^o into a das .ooni on 
one or two occasions. Most researchers agree that such a 
prcK-edure makes it virtually impossible to obtain an adequate 
sample of teacher behaviors from which to generalize about 
an instructor s teaching (Braskamp 1978; Centra 1975; McKea- 
chie 1986; Prater 1983; Scriven 1980; Soderberg 1986), Tlie 
procedure could be \'astly impro\'ed, scholars say, by increas 
ing the number of classroom visits and by visiting all, or most, 
of a teacher's classes, although these correctives would require 
substantially more time and a greater investment of other 
resources (Brandenburg, Braskamp, and Or\' 1979; Braskamp 
1978; Centra 1975; Siodolsky 1984). 

'riiird is the argument that methods used for evaluating 
teaching fail to capture the essence and complexity' of the 
teaching act, McKeachie, for example, observes that teaching 
''involves value judgments, and the means for achieving these 
\'alues is complex. Research has revealed that many variables 
interact in determining faculty effectiveness" (1986. p, 266). 
Determining how these variables interact may prove difficult 
using methods ordinarily employed in faculty' evaluation. 

Fourth is a perception that evaluators are inclined to focus 
too heavily on skills associated with effective teaching (Bui 
cock [1984] calls them "observables"; Scriven [1987] refers 
to rhem as "secondary indicators"). In so doing, critics say, 
evaluators may neglect several more substantive concerns, 
such as the teacher s knowledge of subject matter and his or 
her ability to communicate it to students and to inspire stu 
dent learning, epistemological issues, the relationships 
benveen the prcxresses of teaching and learning and the pre 
vailing institutional and student cultures, and classroom 
d\'namics BukxK^k suggests that formative peer evaluation 
may he usefLil in examining a range of teaching behaviors 
in relation to the contexts in which teaching and learning 
occur 

I-inally, scholars and researchers have tried to explain how 
ohser\'ers (or cameras in classrooms) can affect an instaictor's 
performance, making evaluation of "typical" teaching difficult 
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(Bergman 1980: Rriu 1982; Gage 1961; Hart 1987; Sauterand 
VCalker \T6: Scrixen 1980; Siodolsks' 1984; Ward, Clark, and 
Harrison 1981 ). Gage, for example, obser\*es that many 
leachers feel so threatened by visitation that their "perfor 
mance may depend more on (their) neive than on [their] 
leaching skill ' ( 1961. p. 19). Hart, elaborating on this same 
theme, concludes: 

.\(> outsider, }!(> occ(L<io)ial risitor to the <nigoiug intellectual 
atniummn' of cias.<, can hope to undcn.tand rc^y fully the 
internal />n)ccsses, the codes and interactions, of that com- 
minn'n'. -Vo mere ohsenvr canfnlly n iderstand the roles 
of participation in the class. And this s an important lim- 
itation, for the roles of participation control what happerL^ 
if I a ckL^^ and how such happerdngs ur^' txrceired and 
responded to, Moreoren the roy presence of die obsem^, 
howcrer (luict and irithdrawn, is ati i/itcnvntion that alters 
the situation, changes what is being obserred. As- one /pun- 
dit/ argued "To obseriv a class is actually to obserre a class 
heitigohsenv"{ 198**'. p, 16). 

Oilier teachers, asNX'aril. Clark, and Han ison ha\ e obsened, 
seem to "gel up" for jK^formances in front of audiences 
including observers ( 1981 ), These researchers ha\'e found 
lhai the leachers aiiempted lo invoKe siudenis more activeK* 
in classes when ihe participants knew c)bsei-\'ers were present 
ihun when no one was aware ihe\' v\'ere there. Situations like 
this also miliiaie against evaluation of "lypicar* teaching. 

In a pari'cularly cogent wa\\ Scriven succinctly summarizes 
a number of complaints relative to the accuracy of direct cla.s.s 
room obser\aiion and introduces the issue of subjectivity, 
to which WL' lurn nc.Kt: 

/ ■//-.s/. the risit itself alters the teaching, so that the rL^itor Ls 
not looking at a ratidoni sample. Second, the number of 
l isifs is too small to be an accurate sample from which to 
generalize, civn if it avre a random sample. Third, the lis- 
itor is not devoid of ifidependent perso) nil prejudices iu 
faror of or (tgai)ist the teacher ( p. 10 ). 

Subjectivity 

Closely related to the representativeness, acc iirac\'. and lyp 
icalily issue is the third disincentive, subjectivity. Faculty 
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members often charge that ex'aluation of teaching results in 
suhjecth'c, rather than objective, assessments of their perfor 
mance, and several scholars have looked closely at this issue 
(Aleamoni 1984; Ai'den 1989; Bergman 1980; Centra 1986; 
Drcssel 1976; Edwards 1974; Jones 1986; Mclnt>Te 1978; Prater 
1983; Wood 1977). Other researchers have studied personality 
factors that m^y affect the accuracy of the ex'aluations (Ballard' 
Rcardron, and Nelson 1976; Bulccx^k 1984; Maslow and Zim 
merman 1956; Murray 1975). 

Bergman ( 1980), Centra (1975), and Jones (1986) have 
cited the research findings of scxiologist Talcott Parsc)ns 
( 1954) to support their claims that evaluation too frequently 
is based on ascription rather than achievement, on affectivity 
rather than neutrality, on ditTiisencss rather than specitlcitx / 
on particularism rather than univcrsalism, and on collectivitv 
rather than self. In assessing the effects of these dichotomies, 
Jones ohseiA'cs that the first hciov in each j^air represents a 
jXM\sonal aj^i^roach toward evaluation while the second repres 
ents a more bureaucratic approach ( 1986). Bergman suggests 
that e\'aluators too often cirploy the more personal, or sub 
jectiw. of these approaches ( 1979, 1980). 

The notion that faculty exaluation may not be objectixe is 
I^roi")ai")ly affected tc) an extent by differing epistemologics 
on what "good teaching" is and on the proj^er roles of 
teachers in the process of teaching and learning. Since there 
is a lack of agreement on these matters and probabK' never 
will ix^ complete agreement (Bulcock 1984; Smith and Wal- 
\'oord 1993), it may be necessan'— even desirable— to cxplo'M 
and capitalize on this subjectiveness. As Braskamp ( 1978) and 
Centra ( Ur8) ohserw, there may be advantages in having 
colleagues look at peers* teaching from multiple perspectiws, 
especialK' when the purpcxse is to improve performance, 

\X1iile it is likely that on (xvasion, and obviously regrettable 
when, inappropriate criteria are used in assessing teaching, 
the issue of "appropriateness" seems somewhat less important 
in formative exaluation than in summative, since what is 
apj:>ropriate in one context ma\' not be appropriate for another 
(Bulc(K*k 1984; Stodolsk>' 1984). Still, the perception that eval 
nation might be based on subjective impressions lingers, and 
it nuiy be a rea,s()n wliy faculty are reluctant to participate in 
various methods of formative evaluation. 
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Time, faculty Values, and Institutional Incentives and 
Rewards 

Finally, a major disincentive to faculty participation in forma 
ti\'e peer review invoK'es their academic values and beliefs. 
Two key features of this factor are the time required and the 
nature of their institution s reward and incentive structure. 
Scholars obser\'e that faculty have complained that the time 
required to develop, implement, and take part in programs 
to improve instaiction is excessive and of perceived dubious 
value. Wliile this complaint undoubtedly reflects the senti- 
ments of some faculrv', it is at odds with a finding indicating 
the willingness of faculty' in independent colleges and uni- 
versities to participate in various methods of formative eva- 
luation (Kelg 1991). There is, then, intere.st among faculty 
in colleges and universities where teaching is regarded as the 
primar\' professional role for in.structional improvement 
prc>grams. 

Feu'wcRild argue that most faculty do not care about their 
teaching re.sponsibilitie.s. In fact, many would say, "I could 
he a much better teacher if cjnly I had time** (lA)wman 1984, 
p. 213). Wiat is realK' reflected by this comment is that tacult\' 
generally feel overwhelmed by the variety of things expected 
of tncm — publishing, obtaining grants or contracts, teaching, 
conniittee .ser\'ice, super\lsing individual research projects 
of .students, and .so on. .AJl of this suggests that the time con 
cern actually is driven by the incentive and rew:»rd structure 
of the in.stiiution. Many colleges and universities continue 
to place a premium on research and publication with a con 
comitant devaluation of teaching, which in turn militates 
against faculrv' participation in programs of instructional 
improvement (Carnegie Foundation 1990b; Eble 1988; Fair 
weather 1993; Lindquist 1979; Mathis 1979). It might be 
expected that faculty' would be more inclined to participate 
in in.stmctional impro\'ement programs if college and uni 
versities made a .strong commitment U) teaching through their 
incentive and reward structures. 

In the final analysis, though, as Centra insi.sts: 

Cnlcssfcicult}' members are willing to leave the evaluation 
of teaching to students, who possess ()?ily a limited view, or 
to administrators, who often don't have the time or neces 
sa?y hack\^roioul then they must he willing to invest their 
time in efforts in peer evaluation of teaching ( 1986, p. 1). 
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As substantial deterrents as these factors aui be to discourag 
ing facult)^ involvement in peer review for improving teaching, 
a number of incentives to participate have also been noted 
in the literature. We turn to them in the next section. 



104 



ERIC 



TXS 



INCENTIVES 



VCTiile scholars and researchers have noted a number of factors 
that may detract from the use of peer evaluation of teaching, 
they also have offered several suggestions for enhancing the 
pr(K*ess. In this section of the report, we look at ways the pro- 
cess of formative evaluation might be improved and at how 
it might be made attractive enough to faculrs' that they would 
be willing to de\'elop, implement, and participate in such 
programs. 

The incentives that have been proposed by scholars and 
practitioners are of three general tspes: attituainal and per- 
ceptual, methodological, and procedural. They acknowledge 
the need to change basic attitudes of the faculty. They also 
believe that improving the ways in which methods are 
employed will make the prcxess more credible to faculty. And 
the\' believe there are ways of putting programs in place that 
will help to allay the apprehensions, fears, misperceptions. 
and skepticism of faculty with respect to evaluation in general 
and to formative peer evaluation in particular. An aim of such 
incentives is to increase potential participants' ''comfort lev- 
els" with such programming, so that they will come to view 
it as a natural part of instructional evaluation. 

Attitudes and Perceptions 

At this point, we look again at what were called disincentives 
in the previous section, believing they could become oppor 
tunities if examined critically. Wc treat each issue briefly, fcK* 
using on attitudes and perceptions we believe will need to 
be changed if formative peer evaluation is to become a more 
commonplace profe.ssional activity. 

Academic freedom 

Having classes obser\-ed. videotapes of clas.ses produced and 
analyzed, and course materials ;uul instaictor evaluations of 
.students* academic work examined by colleagues- or admin 
istratcHN - often iwc perceived by faculr\* lus threats to their aca 
deniic heedoin rather than as opportunities for professional 
gro\Mh and development. Somehow faculty must be con 
vinced that peer review i.^ (s necessars' for improving teaching 
as it is for providing feedback on manuscripts planned for 
publication (Kdgerton I98H). It is henetlcial, not intrusive, 
.since colleagues have expertise about teaching Lind learning 
that is theirs alone. As Kdgerton, Hutchings, and Quinkm 
explain; 
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0}i nu)Si annpiiscs, student ratifies arc the ''method of 
choice^' for ei cilucitifig teachhig . . . But there's >tio>'e to 
teaching than irhat's critiqued on student eraluatioti forms. 
What's missing in such eraiuation are precisely those aspects 
(jf teaching faculty are uniquely qualified to observe at id 
judge: . . . those things Lee Shu In? an has in mind when he 
refers ti ) ' 'the pedagogy • of su hsta ) i ce ' ' — that rec|i i i re peer 
perspectii ^es and ret lew ( 199 1 . p ). 

Forniative peer ex'akiation of teaching is a means b\' which 
assessment can be approached in a largely nonjiidgm<?ntal, 
non threatening way. like the comments trusted colleagues 
pres ide on manuscripts faculty plan to submit for publication. 
It may he that faculty are more willing to ask colleagues to 
review their research because colleagues are a step removed 
at that point from a summati\'e accept- reject decision. Such 
is also the intent of ft)rmative peer evaluation, where de\'el 
opment— not de<. ision making— is the ultimate goal. Sum- 
mative evaluation is, of cc^urse. another matter. 

Subjectivity 

F'aculty often charge that evaluation of teaching is subjective. 
.\nd they're right. Thex' may not realize, tlnKigh, that it is really 
not possible for assessment of teaching to be totally objective, 
because there is the ine\ itabilit\' that e\'aluat(M\s' values enter 
into the process. /Vs Pittman and Slate observe: 

A system of lvalues acts cl^ a reference point or standard 
against irhich the selected informatiofi is compared The 
I ety nature of ralues dictates that there is no absolute stand 
ard, hut that a)i indiridual or a group of people determines 
its composition. A corollary of this w that changing thf et'a- 
liniting personnel can {titer the ralue emphases and era 
I nation standards. There is }io way to aivid this situation 
completely: no matter what the form of the eraiuation or 
the criteria on which it is based, a set of i c lines resides at 
the core of the process. This means that in derelopi>ig an 
eraiuation system, special attention must he giieti A antic 
ipating problems that could arise fn mi the ralue bases 
represented, the interaction between the ralue hcL^es and 
the spc 'Cif ic ei 'a lu at ion pn jcedi ires, a>id the p( )ssihilit} ' of a 
t he I > igif ig I all I e base, as 1 1 vll ( is to miti / iz if ig these p( )tet 1 1 ial 
problems { IW), p. H). 




About all thai can be lioped for, they say, is for the academic 
community to adopt general standards upon which to assess 
teaciiers' performance. Tlie specificity of such criteria may 
van' from a single criterion (for example, the college's mis 
sion statement (Pittman & Slate 1989) ) to se\eral (Licata 
1986). In formative peer evaluation, assessing teaching against 
flexible— as opposed to rigid—criteria might be viewed as 
an opportunity rather than a liability, since that flexibility will 
allow for teaching to be described from multiple per*pccti\'es, 
prompting, it is hoped, a professor's self-e\'aluation of his or 
her teaching and self reflection on critical epistemological 
issues. 

Time, faculty values, and institutional incentives and 
rewards 

Facult}' probably will find time for any professional acii\ ity 
if they are convinced it is valuable to themselves and or if 
they are rewarded for it. Because faculty at many colleges and 
universif.ies believe they arc promoted and tenured more for 
their research than their teaching, th.cy likely are to be more 
activeK' engaged in the former than in the latter. Even (hough 
studies show that faculty are more interested in teaching than 
in research (Carnegie 1986; I^dd 19"'9), "some teachers feel 
forced to give up the intrinsic satisfactions of teaching for the 
external rewards of research** (Cross 1988). So. too, probabK' 
with programs of instructional improvement. Ways mu.st he 
found to make institutional incentives and rewards attractive 
enough to faculty that they will make time available for teach 
ing and teaching improvement. The resolve of top level 
administrators with respect to instructional imj^rovement i,^ 
crucial if higher education is lo demon.straic its commitment 
t(^ teaching (Fairweather 1993). Lidd's admonition should 
receive careful ccMisideration by those who govern our col 
leges and universiti( s: 

Tf.iv "tccichhi^ IS. rcscaniy co-^unwtit is hardly a tieir one. 
It hcis occifpted the attention of facalt)' a)id admifu'strators 
in the pc^st and hm inspired )mmeroiis eAperinients to re- 
dress pereei red imbalances. My argument is nothing more 
than that these efforts hare largely failed and that today 
the teaching profession is tyrawiized by a research model 
The intellectual health of both academics and academe irill 
be improred if then* are renetred a)id ultimately successful 
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(■*fl(>^^^ togire recagfiitioii to the training ofstiulcnts in what 
is primarily a tcachiJig profession ( 19"'9, p. 6 ). 

Methodology 

The mcilK)d()lt)gical issues, first raised in ilie section on dis- 
iiuentives. are examined again here witii a view tcnvard turn 
ing tliein into opportunities. W'e treat these issues briefly, 
sinee correctives iia\'e ah'ead\' been alluded to or stated 
explicitly. 

Representativeness, accuracy, and typicality 

Facultv complain, with good reason, that teaching cannot be 
accurately evaluated when classroom obsenation is the only 
method of assessment employed. With that in rninc^, scholars 
almost in\'ariabl\- recommend that several methods be used 
and knowledgeable constituencies be consulted so that all-— 
or at least most—relevant teacher beha\'iors are sufficiently 
sampled. In that connection. Seldin emphasizes that: 

I'ac ulty ci xd uatii )n is a complex process, and }io sif igle 
soiox e of data is adequate. The comhijied appraisals of stu 
dents, colleagues, administrators, a) id the professor s self 
assessment are required for reasoJiably reliable and rat id 
judgments i 1084. p. I5S). 

The melhod,s and procedu.res employed can include direct 
classroom obseiA-atiou as long as the pi'tK'ess also includes 
other metliods. Ft)rmative peer evaluation also could include 
videotaping of classes, assessment of course materials and 
instructor evaluations of the academic work of students, port 
folio aiialysis, review of student evaluations of courses and 
instructors, faculty inten iews of students, and the like, all with 
an aim toward providing informaiion to faculty about their 
teaching that is valid, reliable, cr»xlible, and helpful. 

I-acLilty also complain, with cause, ihat clas.sroom obser 
\aiion is nnieliable when a sii\ijle class is ob.served on one 
or nAo (Kvasions. While it probably is not pt)ssible to sa\ with 
certainty how many visits are required to improve reliability, 
the number should be enough lu coinince faculty that it con 
stitutes a representatix e sample of teacher behaviors, across 
the spectrum of ccrjr.ses lau^i hi. If the sample is too small, thr 
prognim will surely lack credihiility. 

In practice, relevant information might be obtained in three 




or Four \ Lsits Cl3raiulenhurg, Braskamp, and Or\' 19''9). F-or 
insiriK'lional imprcn-emeni puipcxses, Katz and Henr\* recom 
mend regular peer visitations and subsequent feedback—pref 
erably one \'isit per week over a period of one, tvvo, or iliree 
semesters (1988). On occasion, faculty ha\'e attended eveiy 
class meeting (Elbow 1980, 1986; Rorschach and VCliitney 
19H6; Shat/.kyandSilberman 1986). 

'riiere is little doubt that the presence of obscncrs changes 
the nature of the "rypical" teaching-learning situation, par- 
ticularly when classroom obserx'ation occurs infrequently. That 
situation could be improved if obser\'ation were to become 
more commonplace and routine. The same could be said of 
all other methods of evaluation. 

It is probably essential that peer reviewers look at what is 
occurring in all, or most, of a faculty member's courses, 
because faculty employ different approaches from one course 
to anotber. As Shulman observes, there are relationships 
between a teachers degree of understanding of subject matter 
and the teaching styles he or she employs, that "teaching 
behavior is bound up with comprehension and transformation 
of understanding" ( 198"^, pp. 17-18). 'ITie formative peer eva 
luation program described by Elbow, in which a designated 
obser\*er is present at virtually all cKxasions when a teacher 
and his or her students interact over the period of a week, 
could be used to identify a teacher s strengths and weaknesses 
in each course, presenting suggestions for improvement, if 
appropriate ( 1980, 1986). 

There is probably a close connection between charges that 
evaluation fails to capture the complexities of the teaching 
act and that it usually is focused on teaching skills, narrowly 
defined, and too rarel>' on more substantive matters. It can 
be argued ibat facult\', who know what teaching is all about, 
sboutd define the ways in which the essence, nuance, and 
substance of teaching are determined and find ways of using 
that information to help colleagues improve the quality ol 
instruction, lidgerion, Hutchings. and Quinlan, for example, 
tell us that "classroom observation is a desirable practice; it's 
a form of peer review that can indeed address the most sub 
stantive, scholarly aspects of teaching" ( 1991. pp- 6 ). \*ideo 
taping, evaluation of course materials, and lussessment of 
instructor evaluations of students' academic work are also \ ia 
ble methods of getting at what's reall>' important. And these 
are only the peer review factors in the instructional 
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improvement eciuation. 

It also might be argued that faculty and administrators, h\' 
allowing student evaluation of courses and instaictors to 
remain the only real source of information by which teaching 
is assessed, contribute to a system whereby evaluation is 
fo<.*used on perflmctor\^ skills rather than on "a pedagog\' of 
substance." If faculty want a credible system of evaluation— 
for improving teaching as well as for making sensible per- 
sonnel decisions— they must take the lead in creating and 
supporting it (Centra 1986, 1993). 

Procedures 

Up to this point, we have looked at issues raised earlier in 
the report. Now we examine four procedural issues that ha\'e 
not as yet been addressed, at least directly. Tliey are: invol- 
vement of the faculty in the planning of progn ,s of instruc- 
tional improvement, establishment of standards of effective 
teaching, training of facult\' in methods of sup^. vision, and 
faculty as interpreters and integrators of information. 

Involvement of the faculty in the planning of 
programs of instructional improvement 

Nearly all researchers agree that the development of succcssdil 
programs of peer evaluation are dependent on the support 
of the faculty and of top-level administrators (Aleamoni 198*:'; 
Arreola 1987; Austin 1992a; Brock 1981; Freer and Dawson 
1985; Heller 1989; Licata 1986; Razor 1979; Seldin 1984. 1990; 
Skoog 1980; Soderberg 1986). Without the support of the 
faculty, Skoog observ^es that the process is likely to be carried 
out perfunctorily, rather than with a genuine commitment 
to instructional improvement (1980). 

The essence of the argument for eliciting the support of 
the faculty in the development and implementation of pro 
grams of instructional improvement is well stated by Meller: 

A decision to engage in peer supen^ision has to come from 
the peer group itself. The key* is ownership: if teachers do 
not feel they own the project, then they will think somebody 
in the central office has a pet idea that is being forced on 
them([9H9. p. 13). 

Evaluation often is seen as an adversarial relationship between 
those who arc to be evaluated and those wIk) will conduct 
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it. Bixx'k cautions that careful attention: 



he^ii vn to the design of the procedures, to the inclusion 
of teachers in the process of design of the procedures, and 
especially to clear and repeated communication with 
teachers about the procedures. . . . With the reduction of 
threat comes the increased likelihood that teachers will effec- 
tively use eraluation data to make decisions about change 
in their teaching practices ( 1981, pp. 235 -36). 

The rcconinicndations of Brock and Heller have been incor- 
porated into the program of formative evaluation described 
by Freer and Dawson ( 1985). This program, in which a reduc- 
tk)n of the adv ersarial relationship was a primar\' goal, 
includes seven recommendations for de\'eloping and sustain- 
ing the program: 

1. 'I'he commitment of adequate funding; 

2. The invoK ement of as man\' teachers as possible in the 
initial planning stages; 

3- An attempt to arriv e at consensus when program decisions 

are made, so that teachers "buy into * it; 
•1. Teacher involvement in training programs or courses in 

methods of supervision; 

5. Collaboration among participants as the prc)gram is 
implemented; 

6. Involvement of teachers in the monitoring and fine-tuning 
of the program; and 

A separation of the functions of summative and formative 
evaluation. 

Freer and Dawson indicate that the program was received with 
more enthusiitsm than the methods of evaluation that had 
previously been emj^loyed ( 1985). Tliey are convinced that 
involving the faculty in the program planning was a key ele- 
ment in a ]')rogram\s success. 

Establishment of standards of effective teaching 

X'irtually no scholar c)r researcher is audacious enough to 
claim there are absolute standards of c*(Tecli\'e teaching upon 
which to evaluate performance. On thecontrar\\ most are 
likely to acknowledge "that there is no single or best wa\' to 
teach and that what is good practice for one teacher may not 
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be so for another" CElbow 1986, p. 197). Or at least they pay 
lip ser\'ice to that notion. Regardless, the standards issue is 
complex, and it deser\'es caredil treatment. 

Some academicians say enough is known about teaching 
that this information should be used as criteria for evaluating 
teaching (Smith and WaKoord 1993). But others say or imply 
that faculty, through discourse and consensus, should arrive 
at standards by which teaching is to be evaluated (Angelo 
1993; McDaniel 1987; Menges 199-^ Seldin 1980: Weimcr, 
Kerns, and Parrett 1988). And still others believe that, because 
too little is known about the interaction of factors that con- 
tribute to successful teaching, it is counterproducti\'e, perhaps 
not even possible, to use generic standards, maintaining that 
successful teaching is context- hound and influenced by sev- 
eral complex factors (Abrami 1985; Bulcock 1985; Cancelli 
1987; Gray 1991; Mathis 1974; Shulman 1988). 

'riiose who assert that common .standards siiould be 
adopted are likely to support the argument made by Smith 
and Walvoord: 

We bare to either measure teaching or lire with a system 
where it remains unrewarded. So we must gather a panel 
of the best people we hare— - faculty) administrators, experts 
in relerant Jielcis— establish the fairest and most accurate 
criteria we hare, a fid then go ahead. And we do hare some 
criteria. . . . \Xe hare enough research on teaching e.xcel 
lence to begin articulating criteria for certification, and 
h}' startitig we'll get more research ( 1993. P- 5). 

On the other end of the spectaim are those who insist that 
it is inadvisable to set dowii uniform criteria for assessing 
teaching effectiveness. They believe that pursuing common 
standards on which all teachers are to be evaluated is mis 
guided because teaching is content and context s[)ecific, 
dependent on the personalities, backgrounds, and abilities 
of both teachers and students and contingent on a nimiber 
of environmental factc^rs. They would probably insist, like Can 
celli, that: 

A system designed to eraluate this process nmst he flexible. 
It must besensitire to those rariations in teaching that make 
it such an exciting and challenging actiilty. Reducing such 
a ribrant process to a checklist of behariors performed 
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would reduce tcacbuig to a mechanical acL It may be 
a?}^ued, therefore, that the use of a system that allows peers 
to use their analytic processes toscafi, classify, sort, a?id 
resort, the complex a) id unic/ue sets of data presented iti 
each reriew is the only way to do Justice to teaching ( 1987, 

p. p). 

Lying somewhere between these tv.v. extremes is the view 
that faciilr\' should debate and ultiniately agree on the stand- 
ards b\' which their teaching will be assessed. They insist that 
'*facult\' members themselves will have to figure out whether 
and how [general principles of effective teaching) apply to 
their particular disciplines, courses, and students" (Angelo 
1993, p. 3). 

A number of scholars doubt there are enough general prin 
ciples of effective teaching upon which to constnjct viable 
programs of faculty evaluation. For example, Abrami contends: 

// is time to ahand^'u the notion of a single model of effec- 
tire ifistructiofi or the ideal teacher and begin to think in 
interaction terms . . . The question should not be "What 
is the ideal college teacher?"' but rather ''What is the ideal 
college teacher for different contexts (i.e., courses, students, 
and settings) and different t goals, object ires, or desired out- 
comes of instruction?'' (198"^. pp. 223-24) 

Tlie interaction to which Abrami has referred alludes to the 
difficulties involved in e\*aluating teaching. Adding detail to 
that idea, Menges imd Math is observe: 

I'.ffectii eness in teaching depends not o)i a single charac- 
ten^tic hut on the appropriate fit among many variables 
These variables include the purposes of the teaching leaniing 
encounter characteristics and preferences of teachers and 
learners, circumstances of the teaching-learning activities 
and of the larger environmefit in which those activities 
occur, and methods used for detennining success of the 
teachitig and oj the learning. Effective teachers monitor 
and mafiage all of these variables, ensure their consistency 
a fid fashion them into a pkwing whole ( 1988. p. 10). 

Cioming al the same lopic from a different perspectix'c, namely 
course planning. Gray concludes that "the prcKx\ss for iissess- 
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ing the effectiveness of instaiction shc/uld be flexible because 
the situations and contexts of teaching are extremely varied*' 
( 1991, p. 54). In course planning alone, :us Gray indicates, the 
profiess includes "loiowledge luid skill of insiaictor and stu 
dents, unique needs of the discipline, level of students and 
the instaictor, instmctor \s experience, deliver\' mode, depart 
mental guidelines and instaictor controi, environment and 
resources, incentive systems and student motivation, and per 
ceived roles" (p. 54). 

Tlie situations and contexts to which references have been 
made are consistent with Shulman\s "knowledge and learn 
ing" construct, which can provide a useful framework for look 
ing at evaluation of teaching — fonnati\'e peer evaluation in 
particular and other components of comprehensive faculty 
evaluation in general (198*^). Common criteria may be 
appropriate for assessing one category' (general pedagogical 
knowledge) of the knowledge b:Lse. Common criteria may 
he less appropriate for other bases ( knowledge of learners 
and their characteristics; curricular knowledge; knowledge 
of educational contexts; and knowledge of educational ends, 
puq")oses, and values). Such criteria may he inappropriate for 
the remaining bases (knowledge of content and pedagogical 
content knowledge), depending on who assesses the per 
formance and how the assessment is conducted. It seems 
likely that the interactive effects of teachers* knowledge in 
these areas is imique to each indi\'idual. 

Centra is convinced that qualitative methods of evaluation 
are more promising than quantitative methods for examining 
teaching in detail Tliis method is consistent with the \'ie\v 
that evaluation must be adaptable to a varier\' of teaching and 
learning styles. In summarizing his thesis, (xMitra obseives: 

A ciualitatiiv approach wotild itiroli v descriptions oj class 
room iustructiofi hcLwi on the per i eptions of the ohsen vrs. 
. . Descriptiofis hy several ohsen'm will wore likely reflect 
possible personal biases and the residting narratiiv could 
he much more useful [than/ rating scales and numerical 
judgments ( I9H6, pp. 3 4 ). 

Faculty need to consieler the ramifications of the standards 
issue carefully. If they are not willing to do so, it is unlikely 
that e\'aluation, for uiiatewr purpose, will have long term 
credibility. 




Training of faculty in methods of supervision- 
evaluation and of communicating feedback 

Besides claims that faculty wt)uld be more likely to panicipate 
in fomiaiive peer evaluation if they were to receive training 
in methods of super\ision evaluatjon and of communicating 
feedback, there may be additional benefits. The availability' 
of training in these areas may be e ssential if instructional 
improvement programs invt)lving peer assessment are to be 
successful. 

Providing training in methods of super\'ision-evaluation 
has been recommended for many years (Brock 1981; Cancelli 
198"7; Centra 1975, 1986; Freer and Dawson 1985; Heller 1989; 
Mclntyre 1986; Menges 1987, 1990; Mikiila 1979; Root 1987; 
Scldin 1984; Sorcinelli 1984; Weimer, Kerns, and Parrett 1988). 
But except for Perlberg's useful guidelines for training faculty- 
in consultative skills needed for providing feedback from 
videotape playback- feedback, information about such training 
has been largely limited to providing training in how to con- 
duct classroom obser\'ations (1983). 

information about how to transmit evaluation feedback 
from classroom obser\'ation is found mostly in the literature 
in communications, couaseling, psychology, and organiza- 
tional behavior. Brinko\s review of this literature is an excel- 
lent introduction into effective ways of communicating feed- 
back elicited from obser\'ation and from other methods of 
formative peer evaluation ( 1993). 

Another purported benefit of providing training in methods 
of super\'ision evaluation is to improve interrater reliability 
(Cancelli 198"; Centra 1975; Mclntyre 1986; Weimer, Kerns, 
and Parrett 1988). Such training would require, in effect, that 
agreement be reached on standards of effective teaching, lim- 
iting perhaps some of the flexibility called for by several 
scholars and researchers. 

Mclntyre has developed a program for training faculty to 
conduct classroom observation ( 1986). In this program, a 
number of faculty menibers visit the same classroom as a 
group, compare their findings after the visit, and then attempt 
to arrive at a consensus about the strengths and weaknesses 
of the cjuality of the instruction. Mclntyn reports that this 
approach resulted in a considerable reduction in the varia 
bility of the assessment. 

But scholars also note that training in methods of super 
vision will not eliminate all of the problems asscK^iated with 
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imprcning interraier reliability. Centra has obser\'ed that effee 
tiw training programs recitiire more tinie than many faculty 
members are willing to in\'est ( 1986). Bergman concludes 
that "even with training, inappropriate criieria would still be 
(.|uite influential in peer ratings- -it' only unconsciously" ( 1980, 
p. 10). 

While suggesting that teaching consultants are a good 
source for pro\'iding training in superv ision, Sorcinelli ( 1984) 
acknowledges that it could also be pro\*ided by experienced 
faculty. Agreeing, Brock suggests that wh(")e\*er provides the 
training should ha\e expertise in such areas as: 

(iiidio'i is I uil tech) i olog} ', ctbf i ograph] '. gi v ) i ip ciy v icu)i its. 
i)istnictk))uil eraliiation. atthbutio)i theory, ganihig and 
simulatkni computer assisted iristructk))), pcrsona/ized .snx- 
/(^;;/.s- of instriictioN. and pbilosophk's of educatk)n. Hoiverer, 
the attributes of greatest consec/uefice for the coHsultant\< 
effect ire) less may he a commitment to student k'aniing: 
an ahidi)ig curiosity about the relationship between teacher 
student, and subject matter- an euipathic disposition,- a 
knoirledge of local resources: a tendency toward self 
disck)sure: and effect ire inter^)ersonal commiinicatk)n skilLs 
( 1981. p. 239). 

Still another possible benefit of providing training is to in 
crease awareness. By becoming cogni/.ant of the wide range 
of competencies required of erfecti\*e teachers, faculty may 
become both more accurate obsen ers of the teaching of col 
leagues and more insightful about their own abilities as 
teachers (K;it/. and Henr\' 1988; Mikula 19*^9). 

Yet, fcv.' scholars ha\'e addressed the issue of exactly what 
training should he presided. However. Copeland and Jam 
gochian idcntifs* six general areas in which skills should be 
acquired: interpersonal communication skills, use of low 
inference descripti\*e language, problem definition skills, 
classroom obser\'ation technicjues. data analysis technic|ues. 
and skills enabling effective feedback ( 198S, p I" ). Also. 
Menges ad\ises that. 

Some time should he ini vsted in iniproring ohserratk)n and 
feedback skills Iraining should un vr amo)ig t/fher areas, 
use of appropriate paper and poicil forms to (trganize 
obserr(itk)ns, how t(* select i)ifonnatio}i for feedback iiHch 




/\iK-\v itifornuitio)! Jin' the person being ohsen ecl, hoic to 
diJferoitUite desoiptiiv cnul judgmeutal comments while 
gii ifiiifeedh(icL\ cnid huir to deal irith colleagues if the 
sitii(itio)i heco)ties stressful Role playing is a helpful tech- 
Jiicjue for this trainmg. and role play sessions might he stim- 
ulated hy ride(>tapes of teachers who are not members of 
the^roupi 1^>>H*^. pp. 90 91 ). 

Hcvausc leaching and learning arc complex processes, eval- 
uation of leaching is alsc) a complicated endeavor. Solid irain 
ing in methods ofsupemsion and communicaiion appear 
lo be promising means for improv ing the qualiU' of both for- 
mative and summatixe evaluation and for increasing the like- 
lihood faculty wili avail themselves of instructional improve- 
meni i^rograniming. 

Faculty as interpreters atid integrators of information 

The process of instructional improvement would probably 
be enbanced if facult\' were to become interpreters and inte 
grators (»f information prox'ided by students, administrators, 
and .self assessment as well as b\* fellow faculty members. The 
infomiaiion gathered from those sources should come from 
a v ariety of assessment methods. I iowever, the faculty s role 
in evaluation should surely extend beyond this interpretive 
integrative functi; >n. 

Student evaluatioii of faculty teaching performance con 
linues to be employed more often to evaluate instruction than 
any other method (Seldin lOH-t, lW3a, b)9.V ), Most scholars 
have acknowledged that students should be consulted in the 
process but have geiierally argued that students should not 
Iv the only source of inforriiation. 

Many researchers haw studied the relationship between 
student ratiiigs and such \ ariables as class si/e, expected 
course gra(.le, iinie of day. re(|uired vs. elective course, the 
subject matter, and so on (sec. for example. Arubayi 1 19H"| ), 
While there is sonic disagreement as to how these factors 
affeu student ratings, researchers suggest that ihe.se factors 
be taken into account when personnel decisions are made 
an(.lwhen iivsu uctional inijM'o\enient plans are fonnulaled 
(Colien .md McKeachie l^HO; Craig. RedReld. and Gallu/./o 
b)S(); NkKcadiic b)S() ). Cohen and Nkiveachie suggest .spe 
cificaliy that "student ratings .should be evaluated by | faculty] 
peers who know the circumstances uriclcr which a particular 
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u^Lirsc was taught" (p ISl). These- rcsc-aivluTS cmj-)ha.si/c 
tlnit siiidcni ratings cannot he takon at face \aliio. 

Faculty members thcmsclws appcai' to he di\ itlcd on the 
orficacN' of faculty peer re\ ie\v of student ratings of courses 
and instructors. Hritt ( 1982 ). for example, has found that only 
36 jicrcent of tacult\- were in fa\*t)r o( ha\*ing their colleagues 
exainine student ratings. Dienst ( 19S1 ) has found e\en less 
support (33 percent), hut peer review t)f student ratings out 
distanced sujipt)rt for direct cla.ssrot)m ohscnalion antl e\al 
nation of course materials. The riixling.s from these studies 
are difficult to inteipret. ht)we\ er. since it was not made clear 
it respondents heliewd the re\ iew was for summatiw ov for- 
mative jiiirjioses. 

In a more recent studx". facult\- attitudes toward se\eral 
aspects of formative evaluation are examined. Keig has found 
that more than 62 percent of respondents to a survey indi 
cated their belief that instructional e\ aUiation would be 
enhanced by faculty peer re\ iew of student e\ aluatit)ns 
i 1991 ). But this relatively h.igh fa\-orable response is consid 
erably less than that for direct classroom ob;ser\aiit)n. e\al 
uation of course materi lis. and assessment t)f instructt)r 
graded student work, for facult\- re\ iew of student 

evaluations of courses and instructors and for videolaj^ing of 
classes is essentially the same. 

\c\ if faculty are to play a really .signillcant role in the e\ al 
uation ot colleagues" teaching, jieer re\ iew nuist exlentl 
beyond inteqiretatitin of student ratings t)f course> and 
instructors. If faculty are noi willing to evaluate aspects of 
teaching they are uniquely qualified to as.sess. there is a 
tlanger that "what is peer reviewed " is not the process of 
teaching and its products (the learning that the teaching 
enabled), but jmerelyl the obser\ations and ratings submitted 
by students and assorted others ( ridgertt)n. Hutchings. ami 
Quinlan 1991. p S). 

Faculty are clearly qualified to as.sess colleagues' teaching 
at \arious points t)f the process. Thex can evaluate what occurs 
p.rior to and follow ing deli\ en" oi' instruction by examining 
course materials, instructor as.scssments of the academic work 
of students, aiul teaching portfolio.s. The}- can also look at 
w hat occurs when acher antl his or her students interact 
from perspectives ditlerent from that of .sttick'nts. through both 
direct clas.sroom ob.ser\alion and videotapes of cla.s.ses. The 
Ibniialiw peer re\ iew j)roce.ss mu.sl include more than an 
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examination of siudcni ratings of courses and instructors, 
important as that function is. 

As programs of formatixe peer evaluation are being con 
sidered. efforts must he made to develop programs that are 
attractive enough to invite faculty participation. Accomplishing 
that aim involves ad<.lressing a number of faculty concerns: 
intrusiveness. comprehensiveness, object iveness, and insti 
tutional rewards and incentives. Gaining faculty acceptance 
also includes inx'oK'ing them in program design and imple 
mentation, in determiriing the criteria on which their teaching 
will be assessed, in programs [-jroviding training in methods 
of si»per\ision and communication, and in the analysis of 
information provided by students and administrators on 
faculty members' teaching. 

If faculty CLin be con\ inced to participate in formatix'e peer 
assessment programs, there ii apj^ears. benefits for them, 
their student.s. ant.! their institutions. We discuss these benefits 
in the following .section. 
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PERSONAL AND INSTITUTIONAL BENEFITS 



Scht)lar.s suggest that a luimbcr of personal and institutional 
benefits might be realized from facult)' participation in for- 
mative peer exaluation of teaching. Some of their claims are 
based on hunches, others on theoiy. and still others on qual 
itaiive and quantitaiix e research studies. In this section of the 
report, \vc look at the e\'idence supporting and qualify ing 
some of the puipt)rted benefits. >X e examine four areas: 
improvement in leaching: improxemeni in student learning; 
improvement in facult\' morale and in the collegial climate 
of the institution; and improvement in the tenure success of 
junior faculty. 



Improvement in Teaching 

Formative evaluation of instruction has come about largeK' 
bcvause its proponents haxe not been satisfied that summativc 
evaluation actualK" facilitates better teaching. Formative eval- 
uation involving faculty in the assessment of colleagues* teach- 
ing has been developed and implemented because practi- 
tioners believe that facult\' are more qualified than students, 
administrators, and other constituencies to evaluate some 
aspects of instruction, although few contend that peers are 
the best source of information about all aspects of teaching. 

Some programs of formative peer e\*aluation have been put 
into place at colleges and universities where instructional 
development c)tTicials have encouraged its dev elopment, but 
other programs have been organized and developed by faculty 
who are committed to insuuctional improvement. Most faculty 
participating in these programs have expressed satisfaction 
with the results, believing their teaching improved because 
of a.ssislance receiwd from colleagues. 

Wliile there is no particular reason to doubt participants" 
positive assessment of these programs, researchers note that 
evaluation of such prognims often has been limited to self 
reports of participants and too rarely subjected to more rigor 
ous study ( Frickson and Frickson Ur^; Hoyt and Howard 
UrS; jjL'vinson Rose and Menges 1981 ). 

A few relativ ely snuill .senile empirical studies have been con 
ducted of faculty develoiMiient instntctional improvement 
programs (eg.. Flrickson and Frickson 19"'9; Ho\l and Howard 
i9"**9), in which researchers attempted to determine ifstudeni 
nitings of instructors improved when lacully participated in 
the program. In these studies, students have nited the faculty 
member both at midterm and near the end of the semester. 
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with the earlier rating used a5 a covariate in the statistical anal- 
ysis. Tlie results of these studies have shown statistically sig- 
nificant, though niodest, differences between the experimen- 
tal and control groups, with the fornu-r siiowing the most 
impro\*enient between the two rating periods. Wliether results 
from these studies can be extrapolated to formative peer eval 
uation is not clear. Whether improved student ratings of 
instructors actually represent improved teaching also is far 
from certain. 

Levinson Rose and Meiiges emphasize that much of the 
empirical research conducted with respect U) faculty devel- 
opment programs is methodologically weak ( 1981 ). They and 
others recommend that future studies inclut .- larger samples 
of participants, examine the effects of participation over 
longer periods of time, and consider the effects of volunteer 
ism— as opposed to random selection of participants- on 
results. 

Austin's qualitative study on the effectiveness of Lilly 
Endowment Teaching Fellows Program projects is more amhi 
tious in scope than the empirical studies ( 1992:i. 1992h).* 

From her e\'aluation of this particular genre of faciiltv peer 
e\'aluation. Austin notes how the faculty \s teaching improved 
as a result of participation in these programs: 

Fellows often develop a deeper interest ni aud connuitmetit 
to heinga^ood teacher a fid. through their exposure to the 
ones of teaching and learning, cofisciously formulate per 
soualphih <ophies about teaching. For fna ly fellows, the core 
of their emerghig teaching philosophies is a human ism that 
emphasizes appreciation of student differoices, *>!terest 
in listening to students, and a greater comfnitmoit to fos- 
terifig the process of students' intellectual groirth than to 
dispeNsi?ig knowledge. 

*Kr()in Vr \ tliFDUj^li IVH, .Iif pt-riotl of Ausliii s suuly. ShS )iinic>r. noii 
it^iuuvd fat uhv from 3^) rfH-arcfi uni\orsiUt's panioijxucd rn ilu' UHv 
I"nd()\vnicni Tcathm^ I-Vllows Prdj^ram Ofthost- for wlroni mailing 
adtlrosst's anik! hv l(K.'atotl. the restart Iht rct riwd rt'sponscs. in on«' 
form or anotluT (tomprfhcnsrxt- UTiiu-n surwy. hrit-f U'lcfilrom- suni-v. 
short writitMi sur\fy). from pt'opir. k >r «f) pt-n fn\ Data also wt-ri- 
oliiamcd fr(<m pariu ipatrng umwrMtifs' ii;i)i>ram tli.vtiors. ass»KiaU' tlirt-t 
tors, dopartnu-nt t hairs of fellows, and ocIkt iiniwrsiiv administraiors 
"Hk* methods for gatlt<*ring data nu ludt'd rt'vu'win^ archival m.iifn.iK and 
londiKting written and telephoiK- surveys. intei\ e\\s. and i ase studu-s 
l-«M more information ahoiii the studv methotlol >i^\. sft- Austin ftJOJh. 
pp 
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hisi fdlous hare reported that they became viore attenthv 
to students' Icanihig needs: ffiore semittre to such barriers 
as leaniui^ disabilities, a)L\iet}\ arid challenges associated 
with f'jiglish cL^ a second language: and more careful about 
helping students link theoretical coficepts af id practical prob 
lems. This heightened sensitirit}' to students' diverse needs 
and challenges, coupled with exploration of teaching and 
learning theofj. often causes fellows 'o cultivate new 
approaches to their teaching Through group meetifigs and 
individual projects, fellows leani about creative teaching 
methods— the use of computers, games, simulation, and 
cooperative learfiing for example—and ofte?i develop new 
ifistructi(}fial ttiaterialsi 1992a. p. 80). 

Despiic the iniprcncd lai<:hing of tlicse junior faculty from 
research univcrsiiics. success has not come without its costs, 
..\s Austin ohser\es> some faculty have indicated that "the fel- 
io\vsl>ip diverted tim.e fiom research aiid signaled a strong 
ink-re.si in teaching- boih perceived to he negative consid 
eraiions in tenure review [and) frustration with the time that 
hoih excellence in research and teaching requires" ( 1992h, 
p. 89). 

And aj^ixireni success notwithstanding, '\ustin has issued 
three caveats with resi)ect to overgeneruli/ing the findings 
{ I092h, p. 92 ). She explains: 

First, the teachnig fellowship is just ufie element in the 
careers of the participants: thus its impact ls difficult to iso- 
late. Probably most who become fellows already hare c^^/v 
siderable ifiterest i?i teaching a? id aL^o participate in other 
(acuity (Old i)istructio)ial derelopmoit actirities. Second, 
since participatits inrested much time and effort into the 
program, they might oreremphasize its benefits a)id effects 
to justify their commit)iie)it. Third, a c(miparLmi of fellows 
(uid nonfellow:^' career experiences aad teaching- related 
attitudes atid values would have been valuable in idetiti 
fying effects uniijue to the fellowship, but it was not possible 
giroi the tufie constraitits of the study 

oriicials alTiliaied with and external review ers of the Newjer 
se\ MiMcr l-at ulu Program and its successor. Partners in 
Learning, also have told us a great deal ahout how teaching 
improves when facultvtake part in formative peer evaluation. 
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According U) the program manual; 



Through the ohsentitioN process, pahvd faculn^ arc of tat 
exposed to teaching styles quite c/if/erent from their oini. 
Oil a superficial lerel the ohscrrers might focus on tech 
uic/ues that they could apply to their oini classrooms On 
a deeper lei 'cl, the ongoing experience may trigger a self- 
examination that can lead to growth and change more pro- 
found than any classroom technique {yiJlCU. 1991. p. 9). 

Sie\'en Golin, at one time the state program director, explains 
liow students, through inter\'ie\vs with faculty, have moti\'ated 
professors io improxe their teaching: 

Av students reflect on their experiefice. or explain how they 
construct mea/iing we listen, or ask c/iiestiotis The roles 
arerei ersed: They teach we tahe notes. Beyond anything 
the students say— and they are sufprisingly articulate about 
the priorities, their experiences, and their strategies— the pro 
cess of listening attentirely to them affects most of us deeply 
/{njoying their actii e participation a)id excited by the rieiv 
from helou\ we look' for ways to bring some of that excite- 
fuent and enjoyment into the classroom. The fiew relatiofi 
ship with studoits whom we interrieii\ like the relationship 
with our partner, transforms our attitude toward teachifig 
and learning { 1990. p. lOV 

^1 early external program review has also reported on 
impro\'ement in teaching: 



Many changes in techniques and procedures occurred as 
early as the first semester Beyond teaching techniques, how 
erer, faculty hare gained new insights into the learnifig and 
teaching process. Serera I essays /written by program par 
ticipantsj reflected the importance of being demanding yet 
caritig: connecung with and supporting each student. This 
inritational approach to teaching, welcoming studoits to 
become actire participants, can make a difference ( Kice 
and Cheldelin 1989. pp. 1\ 11). 

Rice and Cheldelin also note that "thri)ugh the xarious com 
ponents of the program, faculty haw learned a great deal 
about thcmseKes. Not only arc they becoming more sensitive 




and effective in dealing with the developmental tasks con- 
fronting students, they are discovering a renewed enthusi- 
asm—even passion— for teaching as a vocation" (p. 23). 
On balance, there seems to be a basis for claims that formative 
peer evaluation is successful in improving teaching. More 
generally, as Erickson and Erickson indicate, faculty believe 
that such programs are "useful and well worth the time and 
effort, and that it results in significant, positive, and lasting 
changes in their classroom teaching skill performance" (1979, 
p. 683), It should be obvious, however, that further rigorous 
study of existing programs Is needed before definitive con- 
clusions about program success can be made. 

Improvement in Student Learning 

It is almost unanimously assumed that the true measure of 
successful teaching is the quantity and quality of student 
learning. By extrapolation, it often is assumed that the primaiy^ 
goal of formative evaluation of instruction is to facilitate 
improvements in student learning. While commendable in 
principle, scholars are quick to point to difficulties involved 
in using student learning as a criterion in assessing teacher 
performance and instructional improvement programs. Con- 
cerns of this t\pe are expressed in \'arious ways. 

Scholars emphasize that many factors contribute to, and 
detract from, student k .irning, and only one of these factors 
is teacher performance. Dennis, for example, notes that: 

The effectiveness of a teacher Ls related to a host of student, 
environmental, social and sexual characteristics. [These 
factors] include social-economic status, educational kn^el 
of parents, maturit)^ level of students, the differing expe- 
riences among ethnic groups, peer pressures, and so on 
(1976, p. 440). 

Seldin adds that: 

The quantity and quality^ of student learning are also 
affected hy the student's general academic ability, moti- 
vation to learn, organizing and writing ability on exams, 
skill in multiple choice exams, study habits, and image, 
favorable or unfavorable, in the professor's mind. Eacbfac 
tor affects student achievement ( 1984, p. 122 ). 
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Tliere are, no doubt, other factors that also alfect student 
learning, 

With factors in mind such as those mentioned above, Bui 
coc^k concludes that, among teachers, 

Efforts to measure teaching effectiveness on the basis of 
its impact on student learning Ls unpopular. This is because 
most teachers recognize that student learning is a Tnulti- 
causal activity, and that many significa7it factors . . . fall 
well outside the control of the teacher Thus, to hold teachers 
responsible for the learning behaviors of their students is 
unreasonable (1984, p. 8). 

Elaborating on the theme that much of what students learn 
lies outside of the teacher's pur\'iew, Menges writes: 

Although most teaching occi^rs in the classroom, most learn- 
ing does not. A great deal of learning occurs out of the pres 
ence of the teacher Learning may occur in libraries, labor- 
atories, studios, study rooms, and living arecLS. Indeed, 
learning may occur in any setting where learners encounter 
the subject matter for study 

The Job of the teacher is to be cognizant of all those settings, 
losing them to shape an environmetit conduce, v to learning. 
The essence of teaching Ls the creation of situations in ivhich 
appropriate learning occurs; shaping those situations is what 
successful teachers have learned to do effectively ( 1990 p 
107). 

Scholars sometimes argue specifically that test performance 
and other student products should be used cis measures of 
teacher competence. Some of the difficulties in using these 
materials have been posed as rhetorical questions b\' 
Chickering: 

Do we assess the amount of learning that occurs among 
students, the ''value added'' ifi knowledge, competence, and 
personal development that occurs in our courses? Do we 
evaluate the degree to which knowledge and competence 
demonstrated at the end of a cotirse or the gains that haiv 
occutred are retained? For how long? A semester? A year> 
Vntil graduation? Beyvnd gradu.uion? What are reasonuhle 
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cxpectatioHS coficcniinM the amoinit of learn hi^ ur^ani 
(hat nii^ht occur hi a shi^le course? Ami how do these c^Apc^c - 
tatio}is}ieedtoi wy^ii v}i the prior abiiUw knowledge, com 
petence. a? id exlKrieiices of students? Is a large gahi by a 
reUitireh' poor student to be gireu m>re weight than a small 
gahi by a reiy bright, uvll -prepared student? i I9H4, p. 93) 

Staled simply, "oven \\hen learning outcome information is 
available for'a particular course, we may he unable to deter 
mine what part of tiie outcome should be attributed to that 
particular teacher and course * ( Mcnges 1990, p. 1 1 2 ). 

.\nd with respect to using student outcome measures tor 
evaluating instructional de\elopnient efforts. Hrickson and 
Hrickson conclude: 

// is difficult to deny the attractiivness of student learning 
gains as criteria forjudghig teaching improi ement serrices, 
but we may hare to defer their use as major criteria until 
more practical and powerful eraluation methodologies arc 
arailahle for dealing with the confounding influences of 
textbook's and peers i p. (ri ). 

In cotxiusion. Cross* oi:»ser\ations about teaching and learn, 
ing seem especially apt: 

The ultimate criterion of effect ire teaching is effectire learn 
ing There is simply no other reasoti for teachitig But learn 
iiig probably depends more on //?cd:>ehavior of students than 
on //)c performance of the teacher . . . Good teaching is not 
.so ffiuch a performing act as an erocat ire process. The pur 
pose is to inroire stndent>^ actirely in their own learning and 
to elicit from them their best learning performance ( emjiha 
sis hers) (1991. p. 20). 

if Cross' notions about teaching and learning are to be incor 
poraied into instructional development programs in general 
and formative ju^er evaluation in jxirticular. faculty, admin 
istraiors. and leaching consultants will have to louk at teaching 
in wavs to which many of them are unaccustomed. But by 
examining leaching from this broader perspec tive, evaluaiors 
may be able to reach the essence, nuance, and substaiue ol 
ihis c omplex professional role. 
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Improvement in Faculty Morale and CollegiaUty 

It seems obvious that the purposes for which evaluation is 
carried out would affect faculty morale and collegiality in dif- 
lerent ways. On one hand, .^searchers have found that volun- 
tary faculty involvement in /orm^/r/re evaluation of leaching 
usually affects faculty morale and the collegial climate of the 
institution positively (Austin 1992a, 1992b; Carton 1988; Cross 
1986; Edwards 1974; Freer ^ind Dawson 1985; Heller 1989; 
K:uz and Henr>' 1988; Menges 1985. 1987; Roper, Deal, and 
Dornbusch 1976; Shatzky and Silberman 1986; Skoog 1980; 
Sorcinelli 1984). On the other, they generally have agreed * 
that peer review in the prcKess of .w;/^w^/n;c^ evaluation 
affects morale and collegiality negatively (Brandenburg, Bras- 
kamp. and Ory 1979; Braskamp 1978; Centm 1993; Gunn 
19H2; Mclntyrc 1978; Sorcinelli 1984). 

Despite researchers' findings that morale and collegiality 
will he improved by faculty participation in formative peer 
evaluation, many faculty apparently are unconvinced. Only 
alx)ut 23 percent of faculty from independent colleges and 
universities surveyed believe morale would be improved, and 
approximately 43 percent believe the collegial climate of the 
institution would be enhanced (Keig 1991). In another study, 
Brut has found that about half of respondents did not believe 
facult\' morale would be lowered, but the remainder have 
expressed the opposite view or were not sure ( 1982). Wliile 
the attitudes of faculty are mixed on these issues, it seems 
clear that proponents of formative peer evaluation will have 
to "sell" the benefits of this form of instructional improvement 
to a skeptical Acuity. The following research findings and tes- 
timonials are perhaps a place to start. 

Researchers and faculty menibers themselves have found 
that participation in formative peer evaluation has improved 
the morale of both senior and junior faculty. Froni one study. 
Heller has concluded: 

C 'sing teachers hi a peer supen.^ision role is linked to their 
pmonal growth, tb ir sense of coUegialit}\ and to improved 
instructional practi'x^s-^all of which contribute to higher 
morale, greater job satisfaction, improved school climate, 
and ultimately higher student achievement ( 1 989, p, 1 1 )' 

From his experience as a participant in a program of formative 
peer evaluation, a full professor at a research university notes: 
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To me the project was the balm of Gilead; nothing less. For, 
if I read Erik Erikson convctly, the crisis that a full tenured 
professor of fifty' might face originates in the internal con- 
flict between impulses to remain generative, productive, or 
creative and impulses to stagnate, wither, and dissolve . . . 

So to a professor in his fifties, a personal invitation seriously 
to reconsider his teaching is an invitation to his internal 
forces of generativity> to take heart. Implied in the invitation 
was the suggestion that what had been learned in my private 
experience in years of classroom teaching was worthy of 
public examination. The invitation suggested that work in 
one's classroom and one's thoughts about it desenvd the 
attention of the uppermost echelons of the campus: that as 
an hidividual the professor in the classroom was worth 
training and retraining; that there was a belief in the pos- 
sibility of training him and in the value of what he had 
already learned. Most importantly the project implied tha: 
the students we work with require our serious attention, 
that there is much that we have to learn about them, and 
that we have to discover them if we are to teach them. What 
could be more heartening^ What could resonate with one's 
personal commitments more? To understand et^en more 
fully why! should have been enthusiastic about the project 
we should also consider that a tenured full prof essor has 
nothing to lose HLs position in the university^ is secure By 
honing his skills in teaching the senior full professor adds 
to the areas in which he can exercise leadership (Carton 
1988, pp. 54 S5). 

Junior faculty taking part in the Lilly Teaching Fellows Pro- 
gram, formative peer e\aluation projects put into place at 
research universities, also have found that morale and col- 
legia! ity improved from their involvement. As Austin reports: 

greater self-confidence, self-esteem, and morale have been 
important consequences of the fellowship, identified both 
in self reports hy fellows and in observations by department 
chairs, mentors, and program directors. Some reported that 
the program strengthened fellows' confidence that they could 
handle the multiple pressures of career and family and 
could balance teaching an^^ research rest)omibilities. Others 
appreciated the fellowship experience as a positive part of 
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the tnnisi/iofi froffi ^radiuac stiuloit to faculty mcnihir, 
from uncertainty' to co}iJidcncL\ . . . The feeling that the 
inu'ivrsitv had lurestecJ in them an J cared about their irork 
and careers e?tha}iced co?ifidence and morale for sonie 
( 1992b, p.9-). 

Scholars also address more specifically how collegialit\' has 
been affecled b\' participation in formaiiw peer rex iew. Skoog 
concludes that, through such professional acliviiv, 

I-aculty members acquire knoiiiedge, insights, and strategies 
useful for self superrisioji and self im/)r()iv?ne?it. A/so. cl^ 
a team irorks together supportive relatiofiships are estab- 
lished and discussions concerning teachi fig become more 
commoiL lengthy, and sofMsticated. Ownership of com mo?? 
and unique teachi?ig problems is ack?iouiedged more 
openly. Increased satisfaction and pride i?i teaching ca?i 
result { 1980, p. Z-i). 

Galm has obser\'ed that senior faculty often are highly skcj^ 
tical—even cynical -about programs of instructional impro\e 
ment ( 198S). He reports, however, on how the collegialitv 
ot senior Hnglish facult\- improved by participating in a pro 
gram of peer re\ iew designed by and for themselves, noting 
that: 

W'orkifig irith this post-ten lov study group, I got a amipletely 
dif ferent scfise of my departmou, one that icas i?i ;;/ )• bones 
but, because of the distractions of studoit complaifits, not 
alira ys in tny head. I experienced the power of eight teachers 
and colleagues, a solid core of a permanoit faculty of fifty, 
showing their c(mcern for teaching and demonstratifig their 
accumulated sk'ill It was awesome and a fhw corrective 
fora)\y department chair who may have become cynical 
or hai V lost ifisight of the great faculty^ power in his or her 
(kpartment ip. (D. 

\Xliile S(iK)lars. researchers, and participants alike agree that 
participating in programs of formative peer evaluation can 
lead lo improwd morale and collegiality, they also have indi 
caied thai the mos. successful programs involve the faculty 
in program jManning and iinj^lementation, rely on voluntaiy 
panieipation. and have the unconditional support of top level 
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administrators. Without the unqualified backing of the faculty 
and of influential administrators, instaictional improvenient 
programs are not likely to sustain themselves for any substan- 
tial length of time. 

Improvemeat in the Tenure Success of Junior Faculty 

Proponents of formative peer evaluation of teachmg would 
probably like to believe that participating in it would mprove 
the tenure success of junior faculty-. Evidence supporting this 
proposition is, at this time, inconclusive, and the literature 

on the topic is scant)'. . . . j 

Austin has found that 27 percent of faculty who participated 
in the UUy Teaching Fellows Program as junior faculty at 
research universities believed it had helped them adiieve 
tenure, but 34 percent thought it had no effect (1992b). Austin 
explains that: 

Those who perceived the fellowship as having a positive 
effect on tenure gave several reasom: they enhanced their 
teaching skills and thus their credentials; through Jellows 
meetings, the)' learned about the tenure process; and their 
project work contributed to their publishing record Also, 
according to some senior faculty, department chairs, and 
deans, selection as a teaching fellow appears on a vita as 
a prestigious award. 

Austin savs that those who indicated that participation had 
little effect in the tenure decision believe that: 

Attention to teaching is personally gratifying and topically 
doe^ not hurt tenure credentials unless it « offered as a sub- 
stitute for research activit)'; but their resfwnses clearly indi- 
cate that the qualit)' and quantity of research is thepriman- 
(^me would say sole) factor considered in a tenure deci- 
sion While department chairs and administrators inter- 
'viewed typically praised the fellowship experience and saw 
it as a positive addition to the tenure dossier, they simul 
taneously acknowledged a continuing ^'^""^^ ''"'^''^^'(f"" 
research productivity in tenure decisions ( 1992b, p. ;8 ). 

M research universities, at lca.st, participation in programs of 
instructional improvement hits had, apparently, liulc cttcct 
in achieving tenure. Wlicthcr the same conclusion wcn.ld 
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apply to other rvpes of colleges and universities remains unex- 
am.ned, although 60 percent of faculty' at independent col- 
leges and universities in a Midwestern state believe that junior 
faculty involving themselves in formative peer evaluation 
would enhance their tenure chances (Keig 1991) 

Studies seem to indicate not only /*«/but /jow' teaching 
IS miproved when faculty' avail themselves of instmctional 
development programs in which they work collaborativelv 
to miprove teaching. Studies also appear to show that fa.-ult^• 
mc)rale and collegialit>- improve when faculty are involved ' 
in formative peer evaluation. While student learning mav 
improve when faculty take part in such programs, that is a 
diRicult claim to substantiate, since many variables affect stu- 
dent learning. At this time, there is not enough evidence to 
s'^sjgest that the tenure status of junior faculty i.s enhanced 
w,.en they have participated in formative peer asse.ssment 

Further research is needed with respect to faculty- partic- 
ipation in formative peer evaluation and the puqiorted bene- 
fits ot this involvement. TTiere is also a need for study of how 
tormatn-e peer review is or is not related to life stage and 
career stage theories. 



RECOMMENDATIONS AND CONCLUSIONS 



We have learned many lessons from our review of the liter- 
ature on formati\'e peer evaluation of college teaching. First, 
we have found compelling the rationales for comprehensive 
faculty evaluation, formative evaluation apart from summative 
evaluation, and peer review as part of the formative evaluation 
process. Second, we have learned there are roles in faculty 
assessment that colleagues can perform with distinction but 
also ha\'e learned there are roles they are less-suited to per- 
form. At the same lime, we have discovered that students, aca- 
demic administrators, and teaching consultants have parts to 
plav, each constituency having its own areas of strengths and 
limitations. Third, we have found there are several methods 
of assessment facult>' can use to critique the teaching of col- 
leagues. We have discussed the merits of five of these 
methods but acknowledge there may be other methods that 
also might be employed. 

Fourth, we ha\'e discovered and are discouraged by the fact 
that faculty are reluctant to involve themselves in programs 
of instructional improvement, in part because they are skep- 
tical about its value, because they are fearful of the process, 
and probabh- also because the academy s rewards and incen- 
tives often fail to support such activity'. We believe that the 
academic community will ha\'e to reconsider some of its 
priorities if faculty- are going to be willing to commit time and 
energN' to instructional improvement programs. That said, we 
arc heartened to find there are ways to enhance the likelihood 
of facultN- participation in such programs. 

Fifth, we are convinced that individual faculty, students, 
and colleges and uni\'ersities will beiiefit from faculty par- 
ticipation in fc)rmati\-e e\-aluation. Sixth, we have learned there 
is much vet to be learned about formative peer evaluation. 



Recommendations 

As we read and thought about formative peer evaluation of 
teaching; examined reports describing the nature, strengths, 
and weaknesses of program examples; and reviewed the 
limited critical research on the effectiveness of instaictional 
improvement, certain themes have recun^ed often enough 
for us to gain a sense of what successftil programs are. The 
following recommendations are, for the most part, general 
and broad based: other, more specific recommendations have 
been presented in preceding sections of the report. 
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1. Faculty evaluation should include largely separate 
formative and summative tracks. Summative eva- 
luation, including rigorous quantitative and qual- 
itative data-gathering and analysis, is essential for 
maintaining the academy's integrity; formative eva- 
luation, including equally rigorous descriptive 
strategies, along with ample feedback and oppor- 
tunities for practice and coaching, is necessary for 
improving teaching. 
Since there apparently is little correlation between sum 
mative e\'aluati()n and instructional improvement, strategies 
ihn actually promote better teaching must be identified and 
put into practice. We belie\'e that instmctional developtncnt 
rccjuircs a distinctly different approach to evaluation than the 
practices normally used for making personnel decisions. We 
arc convinced that a truly comprehensiw program of cxalua- 
tion must include two parallel, essentialK' separate tracks: 
summaii\c for decisions regarding reappointment, promotion, 
tenure and compensation; formative for instructional impro\'e 
mcni (and, if desired, for assistance \^'ith scholarship and ser 
\ icc). Our conception of these two forms of evaluation is pres 
ented as Figure 

FIGURE 7 

Comprehensive College Faculty Evaluation 



SumTiat've 
Evaluation 



A Giobal Judgmer.,' 
cl 



jching Scholarship and SofVice 



Critics of higher education frequently remind us that college 
teaching needs to he taken more seriously since it is, in fact, 
**the business of the business." Unless faculty intrinsically are 
motivated by their teaching responsibilities - in addition to 
scholarship and ser\ice or are fascinated with "tinkering 
with leaching,** assuming they know how to fix what isn't 
working, ways must he {\)uni.\ to make instructional dcwl 
opnient activities attractive and usefiil to them, ^X1^ilc these 
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strategies may have to include changes in the institutioiVs 
reward structure, it may be just as important to look for 
approaches that faculty needing and seeking help in improv 
ing their teaching find intellectually engaging. 

2. Formative evaluation should include nonjudgmen- 
tal descriptions of faculty members' teaching by 
colleagues, academic administrators, and, where 
a^'ailable, teaching consultants as well as students; 
each of these constituencies should be asked for 
data only in areas in which the constituency is 
qualified to provide such data. 
We believe that, since teaching is sucli a complex activity, 
it Is unlikely any individual or group can accurately assess the 
full range of a faculty- member s teaching behaviors. For that 
reason, we present a conception of instaictional development 
involving data gathering and analysis by faculty colleagues, 
academic administrators, teaching consultants, students', and- 
uliimaieK'— the professor's self- assessment of his or her teach- 
ing. In our Model for the Formative Evaluation of Teaching 
(Figure 8), bold outlining and connecting lines indicate what 
we consider primary sourc^^s in the /orw^^/'/ e evaluation of 
teaching, lighter outlining and linkages indicate secondaiy 
sources. Tlie importance of these sources probably would 
be quite different in the formati\ e evaluation of research and 
senice and in the summative evaluation of professional 
performance. 

Students are accurate and reliable providers of information 
about many presentational iuspects of teaching but are less 
able to assess an instructor's expertise in content knowledge, 
the qualirv' of course content, and student outcomes. 'Hie 
instructor's knowledge of subject matter, course content, the 
relationship of content in one course to other courses, and 
student achievement should be assessed by colleagues, aca 
demic administrators, and or teaching consultants. If faculty 
are to make accurate self-assessments of their teaching, the\' 
must receive information that is sensible and valid. 

3, Faculty should be encouraged to take part in year- 
long programs of formative peer review of teaching 
every four or five years over the course of their 
teaching careers; that encouragement needs to 
come from administratc^rs and senior faculty. 
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Fruiiflil insiaiciiona! iniprovemeni programs require sub- 
stantial time commitments from and effort of participants, not 
short-term "quic^ •''^xes" or onetime "shots " To h.ave a reas- 
onable chance c ".proving performance, collaborative rela- 
tionships should extend he\'ond a single term of instruction 
and invoK-e periodic formative peer review, not just commit 
tee action when personnel decisions are pending. 

Yet, time is a prec ious commodity, so there may be limits 
to what is productive. In ad\'ocating that faculty- participate 
in intense programs of formative peer evaluation ever\' four 
or five years, we believe we are consistent with Brinko's 
( 1993) finding that "feedback is more effective when given 
frequently, but n(;t excessively" (p. 585). 

4. Faculty should tiike leadership in the design and 
implementation of evaluation programs to improve 
teaching. Although this would seem to go without 
saying, it is Imperative that for such programs to 
be effective, they must come fix)m the faculty. The 
history and traditions of faculty governance of mat- 
ters pertaining to curriculum and instruction 
require that developmental initiatives like this be 
handled similarly. 
To ignore this reality- is to risk either outright faculty rejec- 
tion iind nonparticipation or a cool skeptical reception that 
can slow dovv-n, draw out, and ener\^ate any administration 
imposed plan. This is not to say that there is no important 
role for the administration to play— quite the contrar>\ Hie 
support of the administration is critic al to the success of such 
programs insc:>far as they provide the incentives and rewards 
that underscore the importance of the activity- in the mission 
and life of the instituticm. 

5. Peer participation In formative evaluation involves 
the assessment of several aspects of teaching; peers 
should be trained in the skills needed to conduct 
these assessments. 

Faculty must be knowledgeable in a number of areas v\'hen 
assisting in the improvement of their peers teaching. As eiab 
orated on iii the section on faculty roles, these areas each have 
unicjuc characteristics and skills asscKiated with them. To be 
really helpful to peers, faculty who would he involved in for 
miuive evaluation should be trained in what to look for and 
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how to assist. 

It may be assumed that because one has been teaching for 
a long time, this would not be necessai^'. mile it is true that 
one accumulates experience and a certain folk wisdom about 
teaching in one's field (Shulman acknowledges this in his 
categor)' "pedagogical content knowledge"), it is important 
to test these assumptions against what has been learned in 
recent \'ears from cognitive and developmental psychology. 
Tlie line bem^een experiential wisdom and mythologv' can 
blur. Addressing this in the context of training for assessment 
of specific areas of teaching can help keep these predispo- 
sitions in perspective, 

6. The involvement of peers in the formative evalua- 
tion of teaching should be guided by expertise 
from appropriate areas of the knowledge base of 
teaching; at the same time, care must be taken to 
minimize potential problems that can derive from 
those same individuals' involvement in summative 
evaluation. 

The issue at tlie heart of this recommendation is the poten- 
tial problem presented by a peer participating in formative 
evaluation and later ser\'ing in a judgmental role for promo- 
tion or tenure decisions. The underlying assumption is that 
the two processes must occur separately and that, ideally, no 
one should be part of both processes as an evaluator. This 
must be so, the argument continues, because it is not possible 
to be objective when one has been part of the early prcK'ess 
orobser\'ing and contributing to the "product'' in develop- 
ment. In such a case, the "formative atmosphere^' may be 
tainted by the foreshadowing of judgment to come— that any 
suggestions offered to the faculty member may be interpreted 
as directives, possibly inhibiting the development of unique 
strengths or idciLS in the interest of conformity and sur\'ival. 

Tliis may be an academic freedom question in its lenijrei- 
hcit'di^pca. W\s concern may be compounded by the tact that 
in this time of increasing specialization and, some would say, 
fragmentation, the tuimber of faculty with the background 
to comment upon the content aspects of a colleague's teach 
ing is limited; this would be particularly true of , smaller liberal 
arts colleges with one or tAvo person departments. 

Tliere are three considerations that guide our recommen 
dation in this area. First, we suggest drawing on concepts out- 
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lined and described in Shulman's (1987) knowledge and 
teaching construct, discussed in the introduction of this 
report. By iM'oadening the resources brought to bear in for- 
mative evaluation, the facult\' member receix'ing this input 
can benefit from multiple points of \'ie\\' and expertise that 
rarely resides solely in one person. /\nd since some of these 
areas are outside content specialization, the door is opened 
io faculty outside of c")ne*s field. A number of universities 
around the country- have teaching-enhancement centers with 
consultants and faciilt\' fellows who can help with particular 
aspects of the framework outlined b\* Shulman. 

The second consideration is philosophical. We are among 
the increasing number of those who believe that one cannot 
.separate fact from \alue - that we arc all participants in inquir\' 
and must lake into account the onto! >gical and epistemolog- 
ical constructs and methodological approaches that together 
constitute our point of view. 

The third consideration guiding this recommendation is 
that in the end. wc must rely upon the integrity and **good 
faith" of the faculty. The tradition of faculty governance of 
their own affairs is long and. for the most part, distinguished. 
This is but another area in which factilty must be just and 
caring. 

7. Formative peer evaluation should Include direct 
classroom observation, videotaping of classes, eval- 
uation of course materials, assessment of instructor 
evaluations of the academic work of students, and 
analysis of teaching portfolios. In this process, the 
methods should probably be used in combination, 
not as independent entities. 
We are convinced that factilty are qualified to :i.ssess many 
aspects of colleagues* teaching and capable of assisting peers 
in improving ihcir teaching. To mxximi/.e the benefits of for 
matiw peer a.ssessment. we believe that e\'aluators shotild 
avail them.selves not only of the methods already mentioned, 
but al.so their abilities to inteipret sttident ratings of courses 
and in.su-uctors and rele\ant data from other sotirces and 
conduct interviews with sttidents and colleagties with an eye 
toward examining relationships between teaching .strategies 
and student learning. I \sed together, these approaches can 
provide significant detail about various points of the teaching 
learning pn)cess: where a teacher is preparing to teach the 
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course, where a teacher and his or her students are involved 
in and reacting to the process, and where a teacher evaluates 
students' academic work, provides feedback to students and 
attempts to moti\'ate them, and considers future versions of 
the course. 

a. Peer observation should be an important compo- 
nent of formative evaluation of teaching; however, 
it should not be the only component. 

Colleagues can observe a peer's class from different \'antage 
points than students and other constituencies. Because most 
students attend class more regularly than peer observers, stu- 
dents probably are the most qualified obser\'ers of a teacher's 
presentational skills. On the other hand, faculty^ no doubt are 
much more capable in examining the relationships between 
the nature of the content and delivery and between the con- 
tent of one course and other courses in the curriculum. 

We believe that formative peer evaluation and peer obser- 
x-ation too often are considered s>^nonymous entities. We think 
that formative peer evaluation should be tnore comprehen- 
sive, including a number of methods of assessment besides 
peer obseration. 

b. The videotaping of classes and video playback/ 
feedback should be an integral component of for- 
mative peer evaluation of teaching. It should be 
employed for what it can contribute in its own 
right, not viewed simply as an alternative to direct 
classroom observation. 

Though probably the most self-confrontational of the for- 
mative evaluation methods, video playback/feedback should 
be used in conjunction with other methods of formative eval- 
uation, not avoided. To reduce its often stressful and threat- 
ening nature, we suggest that an instructor view tapes of his 
or her classes with a supportive colleague or teaching con- 
sultant who has been trained in how to interpret what has 
been recorded. Good consultation - kills are rec|uired to rein 
force positive aspects and to diffuse negative reactions that 
can come from \'iewing a recorded performance that is 
deemed less than satisfactory'. 

c. Formative peer evaluation should include exam- 
ination of course syllabi, readings, teacher-made 
tests, teacher-designed papers, projects, presen- 
tations, and other assigned work; professors should 
be encouraged to ask for feedback from colleagues 
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on course materials as they would on manuscripts 
planned for publication. 

Peer evaluation of course materials expands the range of 
teacher attitudes, behaviors, and competencies that should 
be reviewed. Making course materials available to colleagues 
is a way for faculty to demonstrate the quality of course con- 
tent and its relationship to the academic work required of 
students. 

Colleagues from the same field of study or from closely 
related fields are especially well-qualified to assess the appro- 
priateness of materials in relation to particular courses and 
groups of students and in context of departmental and college 
curricula. Faculty also are better able than any other consti- 
tuency to look at relationships betv-'een course materials and 
teaching strategies. We believe that peer evaluation of course 
materials should become a commonplace practice in forma- 
tive peer evaluation. 

d. Peer assessment of instructor evaluations of the 
academic work of students should occur as a cor- 
ollary to peer evaluation of course materials. 

Peer review of instructor evaluations of students' academic 
work can reveal pertinent information about a teacher s com- 
mitment to teaching. A faculty' member's practices in evaluat- 
ing tests, papers, projects, presentations, and other assign- 
ments can tell reviewers much about the prolessof s attitudes 
toward students, especially about how feedback on wiitten 
work is used as a motivatc^r. Peer review of instructor evalua- 
tions of students' work should be used to complement what 
is learned from other methods of formative evaluation. We 
believe that such assessment should be used much more 
extensively in formative evaluation. 

e. The teaching portfolio should be used for self- 
assessment and formative peer evaluation of teach- 
ing; its use should not be restricted to documenting 
performance for personnel decision making. 

Tlie teaching portfolio is an opportunity for faculty to pres 
ent an almost limitless array of materials that dcxaimcnt their 
teaching. Wlien the purpose of a portfolio is instructional 
improvement, faculty should be encouraged to include a 
representative sample of materials, so that a peer reviewer 
can see what an instaictof s typical teaching is like. 

'Hie teadiing portfolio appears to he a promising means 
for evaluating teaching. Its potential for improving teaching 
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8. Institutional rewards and incentives should be 
structured so as to demonstrate to faculty that par- 
ticipation in programs of formative peer evaluation 
to improve teaching is truly valued. 

In the section on disincenti\'es, we ha\'e outlined several 
factors that may attenuate faculty participation in programs 
of this type. Despite the renewed call for an emj')hasis on 
■teaching as **the business of the business/' the implicit mes- 
sage remains— publish. M it has been said, "You have to write 
to sta\' (to get tenured and promoted) and you have to write 
to leave (to be marketable across ''the industr\'''). The prin 
cipal xariablc here, we believe, is time. If facuit\' are made 
to feel that they can invest the time, and it will take time, and 
that it will not come at the expense of their j')r()fe,ssional .sta 
tiis. then the ground is prepared for their participation, 

\C'e would suggest going a step further, however. Not only 
should the implicit threat of punishment be removed, but 
incentives and rewards to participate should be introduced. 
All acrcxss the countiy, at the iastitutional level, .systems of 
financial grants and release time are in place encouraging 
faculty to conduct re.search and to write, A simple measure 
to addre.ss this recommendation would be to .set alongside 
these research awards similar awards for those who would 
devote time to training for and participating in formative peer 
evaluation. The mo.st effecti\'c approach to this significant 
problem, we believe, would be more thorough attention given 
to the multiple disincentives discu.s.sed in the text of this 
rejX)rt. A .sy.stemic as well as a .sy.stematic program rewarding 
liie imj")r()vement of teaching has the be.st chance, we believe, 
to reali/.e the benefits that we have di.scussed in the .section 
on personal and in.stitutional benefits. We believe that most 
faculty are essentially magnanimous and that given a condu 
ci\'e almosj")here that encourages and rewards, rather tiian 
j^unishes. they will be helpful. 

In this connection, we further .sugge.st that .such compre 
liensive j-^rograms take into account lacult\' members' career 
and life .stages. Tliere .sliould be special incentives and 
rewards for the young a.ssi.stant profcs.sor as well as the .senior 
full profe.s.so! I'here .should be recognition of evolving per 
.sonal and developmental changes that can occur over time, 
since mo.st of us change in our intere.sts and attitudes over 




the course of our aireers. Incenth'es and rewards should take 
into account these dimensions of our lives as whole people 
as well. 

9. Research should proceed along several potentially 
lucrative lines. 

First, research needs to be performed about the interaction 
of variables affecting formative peer evaluation in specific 
institutional contexts. VCtiile we punx:)rt to know general con 
ditions that are required for such programs to succeed, dif 
Terences among institutions with regard to governance tra- 
ditk)ns, collectixe bargaining, and other variables need to be 
investigated for what they contribute to successful or unsuc- 
cessful implementation. 

Second, research is needed to tie e\'aluatk)n to motivational 
theor\'. 'Ttiis is to acid support to a similar call from Blackburn 
and others ( 1991 ). "^liile we can speak in broad terms about 
motivation to participate in and to support programs to 
impro\-e teaching, we need to sj^eak more specifically about 
how motivatu)n comes into play. 

Third, we believe there is a need to dcx'ument and more 
widely report the experiences with formative evaluation pro- 
grams where the\' exist, regardless of whether they are sue 
ceeding. Tliere is a paucity of such reporting in the literature, 
as our section on program examples should indicate. Tliere 
may ix* an opportunity here for a national higher education 
organi/.atk)n to facilitate such an effort through conferences, 
a clearinghouse, or use of the Internet. Systematic empirical 
and ethnographic research of initiati\es could contribute 
much to our understanding lls well. 

Conclusions 

No longer are the outcries for reform of higher education 
emanating solely from outside the academy. In fact, some of 
the most strident criticism— as well as the most thoughtful 
comments on improving college teaching are coming from 
the ranks of the professoriate. 

Faculty understand, perhaps better than anyone else, that 
impro\ ing the cjuality of teaching is complicated by many fac- 
tors, all of which enter into their discussions on how it can 
be accomplished. Not the least of these factors is the epis 
temological issue of what "effective teaching ' is. In a sense, 
lack of agreement on this issue supports a flexible, formative 
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approach to assessment, one in which the facult>' play prin- 
cipal roles. From years of experience as students and teachers, 
they know that successful teachers demonstrate not only com- 
mand of subject matter but also knowledge of teaching strate- 
g.os and learning theories, commitment to the intellectual 
and personal development of their students, awareness of the 
complex contexts in which instaiction cKcurs, and concern 
about their colleagues' performance. Perhaps more effectively 
than any other of the academy's constituencies, they can des- 
cribe and analyze their peers' teaching and assist them in 
improving this aspect of professional performance. 

We believe that formative peer evaluation is a promising 
method in which faculty can work cooperatively to improve 
teaching. We are convinced the time has come for the aca- 
demic community to k)ok seriously at its potential, for faculty 
and administrators to collaborate in developing and imple- 
menting this form of instructional improvement, and for 
researchers to evaluate the results in appropriately rigorous 
^'a\*s. 
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